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Abstract 

Over  the  last  five  years,  significant  actions  have  been 
taken  by  upper  management  levels  to  emphasize  the 
importance  of  designing  into  new  weapon  systems  a  higher 
degree  of  reliability  and  maintainability.  To  further 
these  efforts,  research  was  needed  to  first,  measure  the 
significance  of  specific  constraints  impacting  the  "front 
line"  initiators  in  the  process,  namely  the  Deputy  Program 
Managers  for  Logistics  (DPMLs) ,  and  second,  to  identify 
their  recommended  solutions  to  these  constraints. 

A  literature  review  of  what  senior  officials  felt  were 
problems  in  achieving  R&M  initiatives  was  developed  into  a 
list  of  seven  acquisition  logistics  constraints.  The  list 
was  then  evaluated  through  personal  interviews  with  the 
DPMLs  and  Directors  of  Logistics  (DOLS)  of  the  major 
programs  within  Aeronautical  Systems  Division  (ASD)  and 
Electronic  Systems  Division  (ESD) . 

The  respondents  ranked  the  factors  on  a  graphic  rating 
scale  and  provided  additional  constraints  they  believed 
impacted  their  mission.  Statistical  tests  showed  a  clear 
consensus  by  the  respondents  of  the  rank  ordering  of  the 
seven  constraints.  In  addition,  there  were  no  significant 
differences  in  the  rankings  by  the  two  product  divisions 
and  few  statistical  differences  between  the  respondents 
regardless  of  their  rank  or  experience  level. 


To  enhance  the  applied  value  of  the  research,  the 
majority  of  the  interview  time  was  used  collecting  what  the 


DPMLs  perceived  as  the  best  solutions  to  these 
constraints.  Their  comments  were  candid  and  their  37 
recommendations  deserve  further  study,  in  that  they  were 
offered  by  the  people  who  work  closest  to  the  logistics 
acquisition  problems. 


ACQUISITION  LOGISTICS  CONSTRAINTS  AND  RECOMMENDED 
SOLUTIONS:  PERCEPTIONS  OF  SENIOR  DEPUTY  PROGRAM  MANAGERS 

FOR  LOGISTICS 

I .  Introduction 

Problem  Statement 

In  FY  84  the  United  States  Air  Force  spent  over  $20 
billion  in  the  acquisition  of  various  weapon  systems. 

Though  a  large  sum  of  money,  these  acquisition  costs  on  the 
average  will  be  only  40%  of  each  weapon  system's  total  life 
cycle  cost.  Much  of  their  large  life  cycle  costs  will  be 
based  on  each  system's  design  characteristics  of 
reliability  and  maintainability  (R&M) .  Therefore,  the 
Department  of  Defense  could  save  billions  of  dollars  each 
year  as  well  as  improve  total  warfighting  capability  if 
systems  were  designed  with  a  greater  degree  of  reliability 
and  maintainability.  Because  of  these  potential  savings, 
numerous  changes  have  been  made  in  the  last  decade  to  try 
to  improve  the  Defense  Department's  overall  acquisition 
process . 

After  a  long  history  of  investigative  reports  and  blue 
ribbon  commissions.  Deputy  Secretary  of  Defense  Frank 
Carlucci,  who  came  into  office  in  1981,  emphasized  less 
studying  of  the  problems  and  more  implementing  of  solutions 
(5:57).  One  of  his  eight  management  principles  stated: 


Improved  readiness  is  a  primary  objective  of 
the  acquisition  process  of  comparable  importance 
to  reduced  cost  or  reduced  acquisition  time. 

Resources  to  achieve  readiness  will  receive  the 
same  emphasis  as  those  required  to  achieve 
schedule  or  performance  objectives.  Include  from 
the  start  of  weapon  system  programs  designed-in 
reliability,  maintainability  and  support  (18:1). 

In  order  to  do  this,  it  is  necessary  for  the  Air  Force 

to  identify  the  major  constraints  of  designing  the  optimum 

level  of  reliability,  to  determine  their  level  of 

signif icance ,  and  to  develop  methods  to  reduce  the  more 

significant  constraints.  The  principal  people  responsible 

for  initiating  supportability  considerations  for  a  weapon 

system  are  the  Deputy  Program  Managers  for  Logistics 

(DPMLs)  and  the  Directors  of  Logistics  (DOLs) .  Though 

there  are  differences  in  the  scope  of  their 

responsibilities,  references  to  DPMLs  usually  will 

encompass  both  DPMLs  and  DOLs.  Working  through  a  System 

Program  Office  (SPO) ,  the  DPML  has  various  roles,  but 

his/her  primary  one  is  influencing  the  program  director  and 

other  staff  agencies  to  place  more  emphasis  on  reliability 

and  maintainability  during  the  early  phases  of  the 

acquisition  cycle. 

Research  Problem 

If  the  acquisition  community  can  identify  specific 
constraints  impacting  DPMLs,  determine  their  significance, 
and  evaluate  various  recommended  solutions,  a  stronger 
negotiating  capability  for  future  DPMLs  can  be  developed. 


Research  is  necessary  to  accurately  measure  what  the  DPMLs 
perceive  as  the  most  significant  constraints  that  affect 
their  ability  to  influence  u^sign  changes  for 
supportability .  In  addition,  the  DPMLs'  viewpoints  of  how 
these  constraints  should  be  resolved  need  to  be 
consolidated  and  presented  to  decision  makers. 

Though  numerous  articles  and  theses  have  advocated 
stronger  emphasis  on  supportability,  none  have  specifically 
identified  the  factors  constraining  the  DPMLs  in  the  early 
phases  or  offered  recommended  solutions.  Therefore,  this 
research  should  be  valuable  not  only  to  senior  staff 
officers  capable  of  making  the  necessary  internal  changes, 
but  also  to  all  those  who  work  in  acquisition  logistics. 

Background 

Because  of  the  substantial  number  of  regulatory  and 
organizational  changes  made  in  the  last  few  years,  it  is 
important  to  review  this  trend  of  increasing  emphasis  in 
order  to  understand  why  articles  written  over  four  years 
ago  are  likely  no  longer  to  apply  in  evaluating  the  current 
situation. 

The  series  of  studies  and  regulations  that  changed 
acquisition  logistics  began  in  1971  when  DODD  5000.1 
greatly  improved  the  defining  of  the  acquisition  procedures 
(5:56).  In  1972  the  Congressional ly  directed  Commission  on 
Government  Procurement  (COGP)  Report  recommended  numerous 
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changes  to  acquisition  contracting  procedures  (24:19).  In 
1974  the  Office  of  Federal  Procurement  Policy  was 
established  which  centralized  contracting  efforts  in  the 
government.  In  1976  OMB  Circular  A-109,  incorporating  many 
of  the  COGP  recommendations,  placed  more  emphasis  on  design 
concepts  in  terms  of  mission  need  and  established  clear 
lines  of  authority  to  Program  Managers  (24:19).  In  1978  a 
Defense  Science  Board  (DSB)  Study,  "Report  of  the 
Acquisition  Cycle  Task  Force",  made  further  recommendations 
which  resulted  in  changes  to  DSARC  reviews  and  required 
documents  (5:57).  In  1980,  three  regulations,  DOD  5000.1, 
DOD  5000.2,  and  DOD  5000.39,  all  applied  new  emphasis  to 
supportability  in  the  early  acquisition  phases  and  made  it 
an  item  of  interest  in  Milestone  reviews  and  funding 
decisions  (39:5-6).  In  1981  Deputy  Secretary  of  Defense 
Frank  Carlucci  came  into  office.  His  extensive  Acquisition 
Improvement  Program,  consisting  of  32  initiatives,  was 
later  consolidated  by  Deputy  Secretary  of  Defense  Paul 
Thayer . 

In  addition  to  these  regulatory  changes  and  management 
issues,  significant  organizational  changes  were  made  that 
affected  the  Air  Force  acquisition  environment.  In  1976, 
the  Air  Force  Acquisition  Logistics  Division  (AFALD)  was 
established  at  Wright-Patterson  AFB  under  Air  Force 
Logistics  Command  (AFLC) .  It  was  the  first  organization 
dedicated  to  encouraging  decision  makers  to  consider  life 


cycle  costing  (15:10).  In  1977,  the  control  of  the  deputy 
program  managers  for  logistics  shifted  from  the  Air 
Logistics  Centers  to  AFALD.  AFALD  then  began  placing  the 
DPMLs  in  the  Air  Force  Systems  Command  (AFSC)  program 
offices  in  product  divisions  such  as  Aeronautical  Systems 
Division  (ASD)  and  Electronic  Systems  Division  (ESD) .  These 
managers  became  the  primary  interfaces  between  AFLC  and  AFSC 
through  their  attempt  to  manage  the  integrated  logistics 
support  (ILS)  elements  in  each  new  weapon  system.  In  order 
to  do  this,  the  DPML  received  technical  support  from  AFALD 
and  from  AFLC  (28:6). 

In  1978,  AFLC  rstablished  within  its  headquarters  a 
single  point  of  contact  for  acquisition  programs  and 
research  needs  (28:6).  Then,  in  May  1981,  the  level  of 
emphasis  was  further  increased  when  Deputy  Secretary  of 
Defense  Carlucci  published  the  32  initiatives  to  improve  the 
acquisition  process.  Five  Acquisition  Improvement  Program 
(AIP)  actions  addressed  system  support  and  readiness, 
funding  for  test  hardware,  contractor  incentives  for 
support,  logistics  and  support  resources,  and  improved 
reliability  and  maintainability.  In  July  1983,  the  new 
Deputy  Secretary  of  Defense,  Mr.  Thayer,  consolidated  the 
list  of  32  to  6  initiatives  and  support  issues  continued  to 
receive  heavy  emphasis  from  top  management  (4:5). 

In  October  1983,  HQ  AFLC  Commander  Gen  Mullins  and  HQ 
AFSC  Commander  Gen  Marsh  jointly  announced  a  memorandum  of 


agreement  with  three  main  points.  First,  it  changed  the 
AFALD  to  the  Air  Force  Acquisition  Logistics  Center  (AFALC) 
under  the  joint  direction  of  Logistics  Command  and  Systems 
Command.  Second,  it  appointed  AFSC  as  the  leader  in 
acquisition  logistics  policy  making  to  better  integrate 
logistics  requirements  into  the  programs.  Third,  it 
established  a  new  Deputy  Chief  of  Staff  for  Acquisition 
Logistics  within  AFSC  headquarters  and  the  product 
divisions  (37:6) . 

In  January  1985,  Secretary  of  Defense  Weinberger 
decided  to  realign  selected  organizations  and  functions 
within  the  Office  of  the  Secretary  of  Defense.  This  was 
done  in  order  to  "clarify  responsibilities,  strengthen 
controls,  and  provide  emphasis  for  certain  program  areas 
which  should  receive  additional  attention."  (38:1). 

An  Assistant  Secretary  of  Defense  position  was 
established  to  report  directly  to  the  Secretary  of 
Defense.  This  executive  would  devote  full  time  to 
overseeing  the  acquisition  process,  production, 
contracting,  procurement,  maintenance,  supply,  and 
installations  management.  In  addition,  this  Assistant 
Secretary  of  Defense  for  Acquisition  and  Logistics,  ASD 
(A&L) ,  would  serve  as  the  Defense  Acquisition  Executive,  a 
position  previously  held  by  the  Under  Secretary  of  Defense 
for  Research  and  Engineering.  This  would  enable  the  ASD 
(A&L)  to  develop  the  policy  and  provide  the  guidance  on  DOD 


acquisition  and  to  manage  the  procedures  outlined  in  the 

DOD  Directives  5000.1  and  5000.2  (38:2). 

The  decision  has  received  mixed  reviews.  Some  feel  it 

is  good  that  a  single  executive  is  responsible  for  managing 

the  system  acquisition  process  along  with  the  support 

system.  Others  feel  logistics  is  interrelated  with 

personnel  and  are  now  separated,  while  development  and 

production  responsibilities  should  be  separated  because  of 

inherent  conflict  of  interest.  Though  the  position's 

"clout"  has  been  questioned,  most  feel  the  new  incumbent, 

James  P.  Wade,  could  make  substantial  changes  in  the 

treatment  of  R&M  issues  (7:30-34).  Recent  organizational 

changes  were  also  made  at  the  HQ  USAF  level  to  establish 

focus  and  accountability  for  R&M.  These  R&M  advocacy 

changes  were  made  to  offer  technical  expertise,  improved 

program  coordination,  and  greater  R&M  exposure.  In 

addition,  Lt  Gen  Russ,  HQ  USAF  DCS/RDA,  stated: 

Program  reviews  of  reliability  and  maintainability 
factors  will  be  scrutinized  at  all  levels.  Emphasis 
from  the  Inspector  General  and  from  the  R&M  staffs  will 
be  increased.  Independent  review  teams  will  examine 
the  weapon  system  R&M  programs  in  detail  (33:125). 

Similarly  in  March  of  1985,  Air  Force  Systems  Command 

announced  the  formation  of  a  new  Deputy  Chief  of  Staff  for 

Product  Assurance  and  Acquisition  Logistics.  The 

announcement  message  described  its  charter  as  follows: 

The  DCS/PL  is  chartered  as  the  principal  HQ  AFSC 
advocate  to  ensure  that  we  establish  and  successfully 
achieve  reliability,  maintainability,  quality, 
producibility  and  supportability  objectives  and 


requirements  in  all  AFSC  programs.  The  new  organization 
will  formulate  command  policy  that  ensures  these 
requirements  receive  first-line  consideration  in  all 
source  selection,  design,  and  production  activities. 

The  principle  mission  objective  is  to  assure  that 
delivered  defense  systems  and  related  support 
equipment/material  are  of  the  highest  quality,  and  are 
reliable,  maintainable  and  supportable.  The  scope  of 
this  mission  includes  the  entire  spectrum  of  product 
assurance  and  acquisition  logistics  activities  from 
technology  development  to  fielding  of  supported 
systems.  With  heavy  emphasis  on  independent  measurement 
and  assessment  of  our  accomplishments  in  these  areas 
(17:1-2) . 

Due  to  this  increased  emphasis  on  supportability  in  the 
last  four  years  and  its  concomitant  organizational  and 
procedural  changes,  the  literature  review  in  the  next  chapter 
will  primarily  cover  those  articles  published  since  1981. 


Research  Objectives 

The  primary  objective  of  this  research  is  to  identify 
how  the  DPMLs  perceive  the  factors  that  prevent  more 
effective  reliability  and  maintainability  initiatives 
during  the  early  phases  of  the  acquisition  cycle.  In 
addition,  it  will  present  what  the  DPMLs  believe  are  the 
best  possible  solutions  to  these  constraints.  Because  of 
some  similarities  of  this  objective  to  the  1982  research  of 
Major  Hull  and  Captain  Lockhart,  this  research  will  build 
upon  their  findings  and  recommendations.  Their  study 
measured  the  rank-ordered  significance  of  eight 
pre-selected  barriers  to  ILS  as  rated  by  the  mid  level  and 
junior  managers  in  Aeronautical  Systems  Division  (ASD)  and 
Air  Force  Acquisition  Logistics  Division  (AFALD) .  This 
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study  will  expand  the  investigation  from  one  to  two 
product  divisions,  focus  on  the  DPMLs  selected  to  manage 
the  major  programs,  and  narrow  the  purpose  to  impacting 
reliability  in  the  early  phases  of  the  acquisition  cycle. 

Research  Questions 

1.  How  do  the  DPMLs  rank-order  seven  major 
constraints  as  to  their  impact  on  impairing  the  DPML's 
ability  to  influence  a  system's  design  for  improved 
supportability? 

2.  Do  the  DPMLs  significantly  differ  in  their  perception 
of  the  rank  ordering  of  the  seven  constraints  by  product 
division? 

3.  What  other  constraints  do  the  DPMLs' perceive  as 
significant? 

4.  What  solutions  do  the  DPMLs  propose  to  reduce  the 
impacts  of  the  constraints  in  questions  one  and  three? 


II. 


Literature  Review 


Introduction 

To  better  understand  the  environment  in  which  the  DPMLs 
work,  their  job,  and  the  research  objectives,  this  chapter 
will  be  divided  into  five  primary  sections.  The  first, 
"Definitions  and  Relationships,”  will  describe  many  of  the 
terms  used  in  the  acquisition  process  e.g.:  SPO,  DPML, 
product  divisions  and  reliability  and  maintainability.  The 
second  section,  "Initiatives  in  the  Logistics  Acquisition 
Environment,”  will  describe  the  impact  of  the  Carlucci 
Initiatives,  baselining,  organizing  for  logistics  research 
and  development,  and  the  Grace  Commission  Initiatives.  The 
third  section,  "Reliability  and  Maintainability",  will 
provide  an  in  depth  view  of  the  impact  of  R&M  initiatives, 
requirements  definition,  R&M  tasks,  test  and  evaluation, 

RFP  evaluation  criteria,  incentives  to  enhance  R&M 
initiatives  and  the  importance  of  trained  personnel.  The 
fourth  section,  "Measured  Barriers  to  Implementing  ILS," 
will  describe  the  1982  Hull  and  Lockhart  study.  The  fifth 
section,  "Review  Conclusions,"  will  establish  the  base  from 
which  the  methodology  chapter  will  be  built. 

Definitions  and  Relationships 

A  system  program  office  is  a  matrixed  organization, 
headed  by  a  program  manager,  which  is  designed  to  find  a 


solution  to  an  identified  current  or  projected  deficiency 

in  performing  an  Air  Force  mission.  The  selected  solution 

is  frequently  the  development  and  acquisition  of  a  new 

system.  The  SPO  will  explore  various  concepts,  develop  and 

validate  possible  alternative  solutions,  and  then  manage 

the  resultant  new  weapon  system  through  initial 

production.  It  is  the  focal  point  for  all  agencies  in 

system  acquisition  as  well  as  the  only  organization 

authorized  to  negotiate  with  the  contractor.  Therefore,  it 

is  at  this  working  level  that  the  logistics  initiatives 

must  be  integrated  into  the  system's  design  and  the 

constraints  to  designing  into  the  system  the  optimum  level 

of  reliability  and  maintainability  will  occur. 

The  SPO  is  organized  along  program  management 
lines.  Program  management  is  a  special  approach 
to  management.  It  overlaps  the  functional 
management  structure  and  enhances  communications, 
coordination,  and  control.  Program  management 
focuses  on  the  achievement  of  the  end  product. 

The  program  manager  is  the  executive  responsible 
for  all  phases  and  functions  described  by  systems 
acquisition.  Within  the  program  office, 
functional  offices  are  integrated  to  perform  the 
varied  functions  essential  to  acquisition.  Among 
these  functional  offices  is  the  Integrated 
Logistics  Support  Office  (ILSO)  (18:11). 

The  DPML  is  usually  either  a  field  grade  officer  or 

equivalent  civil  servant  who  is  usually  assigned  by  the  Air 

Force  Acquisition  Logistics  Center  to  manage  the  ILSO 

office  in  a  SPO.  His/her  purpose  is  to  interject  into  the 

acquisition  process  the  impetus  to  design  the  system  for 

supportability .  The  DPML's  purpose  in  trying  to  insure 
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supportability  is  to  lower  the  long  range  or  life  cycle  cost 
of  the  system.  Program  managers  in  the  past  have  been 
evaluated  more  on  the  short  run  goals  of  meeting 
performance  standards,  remaining  on  schedule  and  achieving 
minimum  cost.  Therefore,  the  DPML  frequently  has  had  an 
adversarial  role  of  convincing  his/her  program  manager  to 
divert  limited  resources  from  those  other  goals  to  the 
achievement  of  reliability  and  maintainability  design 
enhancements  which  satisfy  long  run  goals.  Like  the  experts 
in  other  functional  specialities,  the  DPML  is  collocated 
within  the  SPO  and  is  responsible  to  the  program  manager. 
However,  as  a  non-AFSC  asset,  the  DPML's  reporting  chain  is 
often  different  (18:11).  His/her  evaluation  is  usually 
closed  out  at  the  Air  Force  Acquisition  Logistics  Center  or 
at  Headquarters  Air  Force  Logistics  Command. 

Similar  to  the  DPML,  the  Director  of  Logistics  (DOL)  is 
usually  a  Colonel  or  equivalent  civilian  grade.  His  purpose 
is  the  same,  however,  his  scope  is  broader  in  that  several 
DPMLs  usually  work  for  him.  He  serves  as  a  staff  member  to 
the  overall  program  director  and  insures  that  all  the  DPMLs 
within  that  SPO  are  supported  and  performing  their  mission. 

The  DPMLs  and  DOLs  in  this  survey  all  worked  in  SPOs 
that  were  involved  in  major  programs.  A  major  program  is  a 
designation  given  by  the  Secretary  of  Defense  to  those 
acquisition  efforts  that  either  involve:  over  $200  million 
in  R&D  and  $1  billion  in  procurement  funds,  special  interest 
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of  the  Secretary  of  Defense,  joint  acquisitions  by  two  DOD 
components  or  another  nation,  or  Congressional  interest. 
Because  of  this  designation,  the  key  positions  in  major 
programs  are  usually  manned  with  the  more  experienced 
personnel . 

Another  organizational  term  is  "product  division."  Air 
Force  Systems  Command  has  divided  up  its  mission  of 
developing  and  acquiring  new  weapon  systems  into  five 
divisions  based  on  the  type  of  systems  being  developed. 
Space  Division  develops  satellite  systems  at  Los  Angeles 
AFS,  Electronic  Systems  Division  develops  communications 
systems  at  Hanscom  AFB,  Aeronautical  Systems  Division 
develops  aircraft  systems  at  Wright-Patterson  AFB,  Armament 
Division  develops  rockets  and  guns  at  Eglin  AFB,  and 
Ballistic  Missile  Office  (considered  a  "division")  develops 
missiles  at  Norton  AFB.  In  this  study,  DPMLs  and  DOLs  were 
interviewed  in  SPOs  of  two  of  these  divisions  -  Electronic 
Systems  Division  in  Massachusetts  and  Aeronautical  Systems 
Division  in  Ohio. 

The  official  definition  for  reliability,  according  to 
Air  Force  Regulation  80-18,  is  "the  probability  that  an 
item  will  perform  its  intended  function  for  a  specified 
interval  under  stated  conditions"  (16:10).  Maintainability 
is  "a  characteristic  of  design  which  is  expressed  as  the 
probability  that  an  item  will  conform  to  specified 
conditions  within  a  given  period  of  time  when  maintenance 


is  performed  by  personnel  having  specified  skill  levels 
using  prescribed  procedures  and  resources"  (30:23). 

Reliability  and  maintainability  are  separate  but 
related  concepts  that  determine  a  weapon  system's 
supportability ,  survivability,  mobility,  and  availability 
(16:10).  Though  perfect  reliability  would  be  too  expensive 
and  practically  unachievable,  it  would  eliminate  the  need 
for  support  since  an  item  would  never  fail.  Therefore,  the 
lack  of  capability  to  design  in  product  reliability  forces 
us  to  design  in  maintainability  (30:23). 

Reliability  is  measured  in  mean  time  between  failures 
(MTBF)  which  is  calculated  by  the  summation  of  the 
operating  times  between  failures  over  a  period  of  time 
divided  by  the  number  of  failures  during  that  time  frame. 
Maintainability  is  measured  by  mean  time  to  repair  ( MTTR) 
which  is  calculated  by  the  summation  of  repair  times  during 
a  period  of  time  divided  by  the  total  number  of 
malfunctions  during  that  period  (30:23). 

The  life  cycle  cost  of  a  system  is  determined  by  fixed 
costs  plus  variable  cost  (LCC  =  FC  +  VC) .  Reliability  is  a 
fixed  cost  and  has  a  direct  effect  on  the  variable  cost  of 
maintenance.  The  higher  the  degree  of  reliability  built 
into  the  design,  the  less  it  will  cost  to  maintain  the 
system.  However,  since  additional  reliability  costs  money 
there  is  a  point  of  diminishing  return  in  which  the 
marginal  gain  of  additional  reliability  will  exceed  its 
marginal  cost  (30:23). 
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As  reliability  increases,  the  life  cycle  cost  of  the 
system  will  continue  to  decline  until  the  marginal  cost  of 
reliability  improvement  exceeds  the  marginal  drop  in 
maintenance  costs.  An  operations  research  analyst  for  the 
Army  Logistics'  Research  Office  states  that  this  optimum 
range,  identified  in  figure  1,  is  not  clearly  recognizable 
and  must  be  negotiated  by  the  decision  makers.  He  proposes 
the  use  of  cost  effectiveness  analysis  (C/EA)  to  determine 
how  much  reliability  and  maintainability  to  purchase 
(30:23-24) . 


Fig.  1  Relationship  of  Reliability  to  Life  Cycle  Cost 


Initiatives  in  the  Logistics  Acquisition  Environment 

In  March  1981,  after  being  in  office  only  six  weeks. 
Deputy  Secretary  of  Defense  Frank  Carlucci  began 
establishing  his  acquisition  improvement  program.  He 
started  his  program  with  a  30  day  comprehensive  review.  A 
small  working  group  of  knowledgeable  acquisition  experts 
gathered  recommendations  from  the  various  affected 
agencies.  After  a  month  of  refining  the  issues  and 
recommendations,  Carlucci  issued  his  memorandum  encouraging 
eight  management  principles  and  31  acquisition  initiatives 
that  later  grew  to  32  (23:5). 

Though  these  initiatives  had  a  "major  impact  on  the 
acquisition  philosophy  and  procedures  and  they  received 
much  publicity,  the  program  did  not  propose  any  new  ideas. 
This  participative  management  approach  merely  took  the  best 
ideas  of  prior  analyses  and  put  them  all  together  in  a 
comprehensive  program  that  emphasized  implementation 
(23:56).  Observers  of  the  acquisition  process  agreed  the 
major  differences  of  Carlucci's  initiatives  to  prior  ones 
were  (1)  his  program  was  broader  in  scope,  (2)  he 
emphasized  implementation,  and  (3)  he  kept  his  watchful  eye 
on  its  progress  long  after  its  introduction  (23:6). 

In  January  1983  Paul  Thayer  replaced  Deputy  Secretary 
of  Defense  Carlucci,  who  returned  to  the  private  sector. 

On  5  May  1983  Thayer  announced  his  consolidation  of  32  of 
Carlucci's  initiatives  into  six  acquisition  improvement 
program  initiatives  (6:17). 
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The  sixth  initiative,  encouraging  improved  support  and 
readiness,  consolidated  the  Carlucci  initiatives  of 
increased  system  support  and  readiness,  funding  for  test 
hardware,  contractor  incentives  for  support,  logistics  and 
support  resources,  and  improved  reliability  and  support. 

The  main  objective  was  to  insure  that  readiness  receives 
the  same  emphasis  as  cost,  schedule,  or  performance.  In 
addition,  a  major  emphasis  was  to  get  readiness  objectives 
established  and  approved  by  the  requirements  validation 
milestone.  The  new  initiative  encouraged  the  separate 
measurement  of  logistics  funds  for  R&M  initiatives  to 
especially  include  supportability  contractor  incentives. 
DODD  5000.1  and  5000.2  have  been  revised  to  reflect  the  new 
emphasis  on  reliability  and  it  is  now  reviewed  at  each 
milestone  in  the  DSARC  process  (6:9-10). 

A  literature  review  of  the  articles  evaluating  the 
effectiveness  of  the  Carlucci  initiatives  shows  a  trend 
from  1981  to  1984  of  two  themes.  First,  "pep  rally" 
speeches  encouraging  the  military,  Congress,  and  the 
industry  leaders  to  get  with  the  program  and  make  it  work. 
Second,  "gloom  and  doom"  articles  stating  this  program  is 
just  more  promises  packaged  differently.  Like  the  wise  men 
describing  the  elephant,  all  were  correct  from  their 
perspective . 

By  April  1982,  one  year  after  initiation,  the  negative 
articles  began  appearing.  Government  Executive  magazine 
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conducted  a  survey  of  24  corporate  executives.  The 
consensus  was  that  the  initiatives  were  excellent,  but  they 
were  "still  mostly  top  level  talk  and  grass-roots  inaction" 
(40:30-32) . 

An  editorial  proposed  that  the  problems  had  become  so 
large  that  they  were  now  literally  impossible  to  solve 
without  direct  Presidential  involvement  (27:46-47). 
Coincidentally,  in  early  1983,  Air  Force  Systems  Command 
held  a  conference  to  review  "how  they  were  doing".  The 
conclusion  was  that  they  were  working  hard,  but  on  the 
wrong  problems  (2:20-24).  A  review  of  the  initiatives  at 
the  end  of  1983,  concluded  that  there  had  been  improvement, 
but  it  was  hardly  big  enough  to  notice  (1:11-12). 

Baselining.  In  addition  to  the  original  32  or  6 
consolidated  initiatives,  several  other  acquisition 
initiatives  have  taken  place  during  the  last  three  years 
that  have  affected  logistics  acquisition.  According  to  a 
1984  progress  report  by  General  Skantze,  then  Vice  Chief  of 
Staff  of  the  Air  Force,  the  problems  are  slipping 
schedules,  changing  requirements,  and  escalating  costs 
caused  by  getting  "off  track  early  in  the  process." 
Therefore,  in  1982,  the  Air  Force  started  a  new  concept 
called  program  baselining  and  cost  capping.  The  program 
directs  "all  the  major  players,  the  operational  command, 
development  command,  support  command,  and  training  command, 
to  accept  and  agree  on,  up-front,  the  requirements,  the 
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program  content,  and  the  cost  of  the  specific  program" 
(34:3-4).  Though  difficult  to  negotiate,  once  all  the 
applicable  MAJCOM  commanders  sign  the  document,  no  changes 
are  made  unless  everybody  agrees  the  changes  are  absolutely 
necessary.  Starting  with  the  B-1B  program,  the  Air  Force 
learned  it  could  adhere  to  a  baseline  price.  In  the  '85 
President's  budget,  baselined  planned  programs  include  the 
B-1B,  C-5B,  Ground  Launched  Cruise  Missile,  MX  missile,  and 
a  few  others  (34:3-4). 

General  -Marsh,  the  commander  of  AFSC,  also  emphasized 
the  importance  of  baselining.  This  means  the  builder, 
tester,  trainer,  and  maintainer  agree  on  such  weapon  system 
aspects  as  schedule,  performance  requirements,  and  support 
details.  "Then  any  change  made  to  the  baseline  will 
require  high  level  approvals.  This  will  help  eliminate 
many  of  the  changes  that  have  historically  caused  program 
cost  growth,  delays,  and  caused  support  funds  to  be  bled 
off  for  mission  requirements."  To  back  up  this  "formula 
for  success,  will  be  sufficient  numbers  of  strategically 
placed  acquisition  logistics  experts"  (37:62). 

Organizing  for  Logistics  R&D .  General  Marsh  stated 
that  another  problem  was  that  no  one  organization  was  in 
charge  of  support  acquisition,  availability,  and 
sustainability  for  combat.  AFSC's  change  in  attitude 
towards  logistics  has  resulted  in  three  new  initiatives. 

First,  since  logistics  research  and  development  is 


showing  big  payoffs,  it  needs  to  be  expanded  and  brought  in 
earlier  to  enhance  reliability  to  even  higher  MTBF  rates 
and  to  reduce  life  cycle  costs.  Current  examples  are 
engines  with  40%  fewer  parts  and  very  high  speed  integrated 
circuitry  (VHSIC)  technology  (37:61).  The  Molecular  Sieve 
Oxygen  Generating  System  on  the  B-1B  will  replace  old 
liquid  oxygen  systems  and  their  extensive  ground  equipment 
thus  saving  more  than  $17  million  (33:124). 

Second,  more  emphasis  is  being  placed  on  the  design 
process  to  ensure  ease  of  maintenance  (like  form/fit/ 
function  components) ,  integrating  new  test  and  diagnostic 
capabilities,  and  insistence  that  equipment  design  include 
testability.  The  B-1B  Central  Integrated  Test  Subsystem's 
elimination  of  flight  line  test  and  support  equipment 
resulted  in  savings  of  $500  million  and  a  60%  reduction  in 
specialized  maintenance  personnel  (33:124).  The  third 
initiative  is  ensuring  all  ILS  elements  are  "funded, 
designed,  developed,  acquired,  and  deployed  before  the 
system  is  delivered  to  the  user"  (37:61-62). 

To  seek  further  enhancements,  the  Air  Force  has 
established  the  Coordinating  Office  for  Logistics  Research 
which  "carries  out  long-range  planning  for  Air  Force 
logistics  research  and  development,  determines  requirements 
in  this  area,  and  fosters  research  that  will  meet  the  needs 
identified"  (31:31).  In  addition,  AFSC,  AFLC  and  their 
subordinate  AFALC  also  play  a  role  in  R&D .  AFSC  "provides 


half  of  the  Coordinating  Office's  manpower  through  'dual 
hatted'  engineers."  Then  requirements  are  packaged  and 
coordinated  with  these  agencies  and  inserted  into  "the 
laboratories'  planning  and  programming  cycles"  (31:32). 

The  Logistics  and  Human  Factors  Division  of  the  Human 
Resources  Laboratory  is  also  a  key  player.  The  Air  Force 
Human  Resources  Laboratory  "conducts  research  and 
development  relating  to  human  factors,  management  science, 
and  operations  research  and  thus  complements  work  in  the 
hardware  and  physical  sciences  laboratories."  Technology 
transition  is  also  important  as  it  "closes  the  research 
loop  by  identifying  specific  users  and  assigning 
responsibility  for  application  of  research  results" 

(31:33) . 

The  Joint  Logistics  Commanders  established  in  1984  a 
Joint  Technical  Coordinating  Group  to  evaluate  each 
service's  laboratory  efforts  concerning  logistics  -  R&D  to 
more  efficiently  use  the  R&D  funding.  In  addition  the  DOD 
directed  the  Air  Force  to  program  approximately  $40  million 
per  year  in  1984  and  1985  for  logistics  R&D  (7:32). 

Grace  Commission  Initiatives.  In  1984,  Paul  Thayer 
responded  to  the  Grace  Commission  review  of  Defense 
Acquisition  by  admitting  reform  of  the  process  will  be 
difficult  and  expensive.  The  Commission  made  275 
recommendations,  of  which  DOD  agreed  with  about  70%, 
implementing  many  immediately.  However,  recommendations 


which  contained  80%  of  the  estimated  savings  required 
congressional  actions.  In  addition,  the  program  stability 
sought  in  the  initiatives  was  also  hurt  by  Congress 
cutting,  reducing,  and  stretching  out  programs.  Congress 
also  restricted  the  power  of  DOD's  principal  tool  to 
stabilize  programs  by  reducing  multi-year  procurements  and 
economical  order  quantities.  By  approving  only  7  of  14  FY 
84  multi-year  candidates,  $1  billion  in  potential  savings 
was  lost  (  36  :  2-4 )  . 


Reliability  and  Maintainability 

In  a  special  study  on  R&M  improvements  prepared  for  the 
Vice  Chief  of  Staff  of  the  Air  Force,  an  air  staff 
logistics  plans  and  programs  analyst  stated,  "Reliability 
and  maintainability  have  significant  impact  on 
supportabil ity ,  survivability,  mobility,  and  system 
performance  and  availability"  (16:10).  Support  costs  are 
brought  down  when  the  reduced  failures  lessen  the 
requirement  for  spares,  equipment,  and  personnel  which 
further  reduces  training  and  facility  costs.  Survivability 
is  increased  two  ways:  first,  through  less  dependence  on 
vulnerable  ground  facilities  and,  second,  through  enhanced 
mission  protection  equipment  such  as  electronic  counter¬ 
measures  and  navigation.  Mobility  is  enhanced  by  R&M 
reducing  the  numbers  and  types  of  equipment,  personnel,  and 
spares  necessary  to  deploy  in  support  of  an  operation.  A 
Rand  study  showed  how  a  reduction  in  F-15  line  replaceable 


units  would  have  deleted  the  mobility  requirement  of  22 
pallets  of  cargo  and  40-50  maintenance  personnel  per 
squadron.  System  performance  and  availability  would  be 
enhanced  through  reduced  repairs  making  more  aircraft 
available  for  missions  and  peacetime  training  sorties 
(16:10-11) . 

Another  way  of  illustrating  the  impact  of  reliability 
is  that  if  in-commission  rates  of  aircraft  can  be  doubled, 
the  number  of  aircraft  capable  of  being  committed  to  battle 
is  also  doubled,  which  is  much  cheaper  than  buying  twice  as 
many  aircraft  (19:13). 

Requirements  Definition.  Decisions  made  in  the  early 
concept  exploration  and  demonstration  validation  phases 
drive  the  cost  and  characteristics  of  a  system  for  its 
entire  life.  By  the  end  of  the  concept  phase.  Milestone  I, 
70%  of  a  system's  life  cycle  costs  have  been  determined. 

By  Milestone  II,  85%  of  life  cycle  costs  are  committed  even 
though  the  funds  affected  will  not  be  spent  until  years 
later.  These  early  decisions  determine  later  support 
concepts  and  system  R&M  characteristics.  Thus 
supportability  problems  are  often  locked  in  before  even 
being  identified.  This  problem  is  compounded  when  R&M 
testing  and  design  improvements  are  deferred  due  to  cost  or 
schedule  constraints  (16:11). 

The  most  important  consideration  during  the  concept 
phase  is  developing  meaningful  requirements  in  the 


statement  of  need  (SON)  and  the  contract  statement  of  work 
(SOW)  (16:13).  A  system's  R&M  requirements  are  established 
primarily  by  the  user  commands  (11:25).  A  new  Air  Force 
objective  is  that  HQ  USAF  will  set  resource  goals  that 
relate  directly  to  mobility,  manpower,  and  life  cycle  cost 
requirements.  These  will  then  be  monitored  against  firm 
program  requirements  at  the  major  commands  (33:125).  Past 
SONS  have  been  vague  in  defining  R&M  requirements  by  using 
comparisons  to  existing  systems  or  generalities  instead  of 
using  meaningful,  quantitative  methods  to  derive,  state, 
and  measure  analyzed  needs.  Likewise,  the  statements  of 
work  tend  to  be  copied  phrases  from  the  SONS  (16:13). 
Unfortunately,  staff  personnel  who  derive  requirements  may 
not  be  aware  of  all  the  R&M  implications.  How  a  failure  is 
defined  is  very  important.  The  difference  in  interpreting 
MTBF  as  only  "hardware  failure"  or  "regardless  of  cause 
failure"  can  have  a  significant  impact  on  the  overall 
effectiveness  of  a  system  once  it  is  fielded  (11:25-26). 

Some  believe  the  major  problems  in  the  early  phases  are 
contracting  restrictions  and  military/contractor  mindset 
constraints.  In  an  effort  to  change  these  issues.  Deputy 
Secretary  of  Defense  William  H-.  Taft  issued  two  memoranda 
in  January  and  December  of  1984  establishing  the 
"Streamlining  Initiative."  This  initiative  calls  for  more 
contractor  initiative  and  a  creative  partnership  with 
industry.  The  principle  goals  are  to  preclude  untailored 


application  of  specifications  and  the  increased  use  of 
specifying  results  required  rather  than  detailed  "how-to 
procedures"  (14:6-7). 


Reliability  and  Maintainability  Tasks.  Since  all 
programs  experience  funding  limitations  and  tight  schedules, 
only  the  priority  items  will  be  fully  funded.  There  is  no 
single  best  R&M  priority  list  since  each  program  is 
different.  However,  at  a  1981  Air  Force  Systems  Command  R&M 
Workshop,  a  panel  composed  of  25  R&M  experts  from  major 
programs,  depot  operations,  policy,  and  research  and 
development  organizations  ranked  the  R&M  tasks  in  Table  I  as 
what  they  perceived  as  the  most  cost  effective  (11:25): 


TABLE  I 

Most  Effective  R&M  Tasks  as  Perceived  by  AFSC  R&M  Workshop 


Reliability 

Maintainability 

1.  Parts  Derating 

1.  Accessibility 

2.  Parts  Selection  and  Control 

2.  Testability 

3.  Failure  Analysis  &  Corrective 

3.  Logistics 

Action 

Supportability 

4.  Parts  Screening 

5.  Burn-In 

A  similar  study  done  by  Rome  Air  Development  Center, 
determined  that  certain  factors  were  common  in  programs 
whose  field  reliability  exceeded  their  stated  requirement 


and  that  certain  factors  were  common  in  those  which  failed 
to  achieve  their  stated  requirement.  The  reliability 
successes  all  had  meaningful  reliability  requirements,  Air 
Force  and  contractor  emphasis  on  reliability,  and  excellent 
parts  control  and  test  programs.  The  failures  had  complex 
systems,  rigorous  environmental  requirements,  routine 
reliability  emphasis,  mandated  off  the  shelf  hardware,  and 
software  problems  (11:25). 

After  the  requirements  are  established,  one  of  the 
first  tasks  is  deciding  on  the  acquisition  strategy  of 
design,  commercial-off-the-shelf,  or  a  combination  of 
both.  Off  the  shelf  procurement  saves  in  initial  cost  and 
time.  However,  this  commercial  equipment  may  not  withstand 
the  military  environment,  may  create  configuration 
management  problems  if  the  manufacturer  changes  design,  may 
be  lacking  in  available  spare  parts  and  maintenance 
manuals.  In  addition  it  has  been  shown  that  extensive 
usage  of  commercial  parts  locks  the  Air  Force  into  one 
supplier,  which  results  in  higher  prices  and  can  also 
threaten  readiness  if  the  contractor  discontinues  the 
product  line.  To  evaluate  all  these  situations,  a  strong 
parts  control  board  or  parts  advisory  group  should  review 
the  contractor's  preliminary  parts  list.  In  Air  Force 
weapon  systems,  about  90%  of  the  electronics  can  be  handled 
with  current  technology  or  previously  approved  parts. 

These  boards  will  emphasize  proven  parts,  materials,  and 
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processes  where  possible.  Knowledgeable  parts  engineers 
can  be  requested  for  these  boards  from  the  Defense 
Electronic  Supply  Center  and  other  various  sources  within 
the  DOD  (11:26-28) . 

In  addition  to  parts  selection,  another  R&M  priority  is 
parts  derating.  Derating  is  reducing  the  electrical, 
mechanical,  and  environmental  operating  stresses  well  below 
the  maximum  level  a  part  can  sustain  in  order  to  increase 
the  lifetime  of  the  part.  Though  there  is  no  recognized 
derating  standard,  there  is  a  document  now  available  from 
Rome  Air  Development  Center  that  allows  an  Air  Force 
reliability  engineer  to  compare  a  contractor's  proposed 
derating  criteria  with  those  recommended  (11:28). 

Test  and  Evaluation.  During  the  design  phase,  early 
emphasis  on  maintainability  will  encourage  using  standard 
and  proven  designs,  fail-safe  features,  sufficient 
well-placed  test  points,  and  worst-case  design  techniques. 
Testability  is  the  capability  to  accurately  detect  and 
locate  failures.  An  early  involved  maintainability/ 
testability  program  will  insure:  (1)  that  testability 
requirements  are  based  on  operational  requirements,  (2)  an 
optimum  mix  of  built-in-test  (BIT)  and  external  test 
equipment  (ETE)  is  selected,  and  (3)  the  testability 
responsibilities  for  the  government  and  contractor  are 
clearly  identified  (11:28-29). 

Testing  does  not  mean  the  DOD  engineers  have  to 
evaluate  the  contractor's  design.  The  government's 


responsibility  is  to  evaluate  the  reliability  of  the  product 
through  both  developmental  and  operational  testing.  Analysis 
of  the  data,  by  an  independent  development  evaluator, 
operational  evaluators,  and  the  user  is  essential  in 
determining  an  accurate  reliability  of  the  system.  However, 
the  different  evaluators  will  often  give  different  analyses 
which  the  decision  maker  must  evaluate.  The  different 
analyses  will  occur  mostly  due  to  honest  differences  in 
opinion  of  what  constitutes  a  failure  and  because  of 
different  but  equal ly -acceptable  analytical  techniques 
(21:26-27) . 

In  1983  the  chief  executive  officer  for  Northrop 
Corporation  felt  R&M  could  best  be  achieved  through  three 
steps.  First,  place  greater  emphasis  on  R&M  during  the 
concept  and  development  phase  during  questions  of  tradeoffs. 
Second,  invest  more  up  front  in  test  hardware  and  apply  a 
strict  testing  procedure  called  "test,  analyze,  and  fix." 

This  procedure  focuses  on  correcting  early  failures  under 
realistic  conditions.  Third,  to  project  requirements  or 
predict  results,  use  plans  based  on  new  technologies  as 
opposed  to  those  based  on  previous  weapon  systems  (19:14-16). 

RFP  Evaluation  Criteria.  Reliability  begins  during  the 
design  phase.  Some  argue  that  engineers  need  to  be  better 
trained  in  the  reliability  studies  of  mechanics,  strength  of 
materials,  thermodynamics ,  digital  circuitry,  and  micro¬ 
processing.  However,  most  engineers  believe  they  know  how  to 


"design  in"  reliability  but  are  not  in  a  position  to  set 
their  own  priorities.  In  fact,  management  determines  the 
degree  of  reliability  instead  of  designers  (21:25-26). 

Three  regulations,  DOD  5000.1,  DOD  5000.2,  and  DOD 
5000.39,  all  strongly  emphasize  that  management  must  give 
equal  consideration  to  life  cycle  cost,  supportability , 
and  systems  readiness  from  the  beginning  of  the  acquisition 
cycle.  However,  past  Requests  for  Proposal  (RFPs)  and 
their  subsequent  contract  awards  have  shown  much  more 
weight  applied  to  cost  and  performance.  In  a  survey  of  top 
managers  of  defense  related  industries,  contractors  were 
asked  what  percentage  weighting  they  thought  was  applied  to 
supportability  areas  in  awarding  contracts.  Two  thirds  of 
them  estimated  it  as  insignificant  or  less  than  10%. 
However,  the  emphasis  appears  to  be  changing  in  recently 
distributed  RFPs.  More  than  90%  noted  the  increased 
emphasis  citing  added  requirements  for  1 if e-cyc le/ support 
cost  analysis.  Yet,  the  respondents  unanimously  believed 
that  if  a  funding  limitation  required  a  cutback,  the 
supportability  area  would  still  be  the  first  to  be  cut  back 
(39:5-6) . 

One  new  program  to  raise  the  Air  Force  consciousness  of 
reliability  and  maintainability  is  R&M  2000.  It  was 
instituted  in  February  1985  when  the  Assistant  for 
Reliability  and  Maintainability  was  created  under  Lt  Gen 
Marquez,  HQ  USAF  DCS/LE.  The  program's  personnel  will 


advise  the  Air  Force  Systems  Acquisition  Review  Council 
(AFSARC)  if  the  weapon  systems  the  Council  is  considering  do 
not  meet  some  "as  yet  undecided  minimum  standards  for 
reliability  and  maintainability."  Though  some  are  skeptical 
of  the  program,  Lt  Gen  Marquez  believes  it  will  be  effective 
"because  of  the  AFSARC  connection's  ability  to  stop  program 
approval"  if  the  R&M  issues  are  not  weighed  heavily  enough 
in  the  RFP  or  its  evaluation  (32:42-44). 

To  aid  in  weighing  R&M  more,  RFPs  will  have  to  clearly 
identify  to  contractors  how  supportability  factors  will  be 
weighed.  Similiarly,  DOD  should  award  contracts  for 
innovative  approaches  for  supportability  (39:7).  One 
possible  way  to  avoid  restricting  potential  R&M  improvements 
yet  still  control  the  program  is  to  state  the  minimum 
requirement  and  then  make  it  clear  a  contractor  can  improve 
his  selection  score  by  proposing  a  design  that  exceeds  those 
requirements.  This  encourages  contractors  to  perform  cost 
tradeoff  studies  during  the  proposal  stage  (35:135-136). 

When  an  individual  buys  a  car  he  can  determine  cost, 
riding  comfort,  power,  and  appearance.  On  the  other  hand, 
since  he  cannot  determine  reliability  until  after  he  has 
purchased  the  vehicle,  he  tends  to  give  it  less  weight  in 
his  purchase  decision.  Similarly,  the  Air  Force  has  tended 
to  emphasize  physical  and  mission  performance  charac¬ 
teristics  in  our  choice  of  systems.  Aware  of  this 
phenomenon,  contractors  will  not  give  reliability  a  high 


priority  unless  they  know  that  independent  reliability 
testing  is  to  be  conducted  and  production  contracts  or 
incentive  awards  depend  on  the  test  outcome  (21:26). 

Low  emphasis  on  reliability  is  not  peculiar  to  the 
military.  Two  similar  articles  discussed  how  U.S. 
corporations  place  less  emphasis  on  product  reliability 
than  Japanese  firms.  Schools  and  U.S.  firms  are  beginning 
to  teach  that  quality  is  no  longer  a  tradeoff  of 
performance  and  that  quality  can  be  a  key  competitive 
strategy  (8:8).  As  a  result  of  what  we  have  learned  from 
Japanese  industries,  more  emphasis  is  being  placed  on 
reliability  engineering  in  early  product  design  (12:71-72). 

Incentives  to  Enhance  R&M  Initiatives.  A  significant 
method  in  incentivizing  reliability  is  through  the  use  of 
award-fee  contracts  which  provide  bonus  payments  over  and 
above  the  contract  price  when  specific  goals  are  exceeded. 
These  goals  can  be  mean  times  between  failures,  mean  times 
to  repair,  factors  of  operational  availability,  or  other 
quantifiable  measures  (35:139).  Two  recent  examples  cited 
were  the  Air  Force's  F-16  and  the  Navy's  F/A-18  programs. 
The  largest  and  most  innovative  one  was  the  Navy’s  program 
that  earmarked  $39  million  for  contractor  incentives  to 
reduce  operating  and  support  costs.  There  was  a  $15 
million  life  cycle  cost  incentive,  a  $12  million 
reliability  incentive  determined  by  mean  flight  hours 
between  failures  at  1,200  and  2,500  flight  hrs.,  and  a  $12 
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million  maintainability  incentive  determined  by  maintenance 
man  hour  requirements  at  1,200,  2,500,  and  9,000  flight 
hours.  To  insure  designs  capable  of  attaining  the  awards, 
the  McDonnell  Corporation  collocated  their  logistics  and 
design  engineers.  Likewise,  the  Northrop  Corporation 
established  incentive  programs  with  34  of  its  equipment 
suppliers.  The  bottom  line  of  all  R&M  emphasis,  funding, 
and  incentives  must  be  the  reliability  of  the  system.  In 
1983,  the  F/A-18  had  a  maintenance  man-hours  per  flight 
hour  of  only  26.2  which  was  much  better  than  the  56.0  hours 
for  the  F-4  and  45.4  hours  for  the  A-7E  (20:3-7). 

Trained  Personnel.  A  General  Accounting  Office  (GAO) 
report  stated  that  other  acquisition  constraints  include 
the  military's  assignment  rotation  policies  and  a  career 
path  "lacking  structure"  which  is  evidenced  by  the 
insufficient  training  offered  and  the  few  incentives  for 
career  commitment.  The  unmanaged  "career  field’s  need  for 
professionalizing"  has  received  recent  attention  from 
Senators  Roth  and  Cohen  who  believe  many  of  the  acquisition 
problems  are  people  problems  rather  than  system  problems. 
Senator  Quayle  recommends  "two  separate  but  equal  career 
paths"  for  operations  and  acquisition  in  order  to  train  and 
maintain  a  corps  of  skilled  personnel.  Though  others 
believed  the  operational  background  can  provide  substantial 
insight,  most  senior  military  leaders  and  Congressmen 
agreed  with  the  GAO  report  that  more  training  and 
assignment  controls  are  necessary  (9:16-25). 


Measured  Barriers  to  Implementing  ILS 

In  1982,  Major  Hull  and  Captain  Lockart  researched  the 
barriers  to  fully  implementing  integrated  logistics  support 
(ILS)  in  system  acquisition  as  perceived  by  ILS  managers 
and  program  managers  in  the  Aeronautical  Systems  Division. 
The  Hull  and  Lockhart  study  considered  all  phases  of  the 
acquisition  cycle  as  opposed  to  just  the  early  phases,  and 
it  considered  barriers  to  the  entire  scope  of  ILS  as 
opposed  to  just  making  design  changes  for  R&M 
improvements.  Likewise,  the  Hull  and  Lockhart  study 
concentrated  on  measuring  the  perception  differences  of 
senior  and  junior  level  managers  and  the  perception 
differences  of  the  ASD  program  management  personnel  versus 
the  AFALD  logistics  personnel.  Though  different  in  these 
respects,  many  of  the  constraint  factors  are  common  and 
thus  their  study  provides  a  base  from  which  to  expand. 

Their  research  concluded  that  there  was  no  significant 
difference  in  how  the  junior  and  senior  level  managers 
perceived  the  relative  ranking  of  the  eight  factors 
(18:68-69).  Based  on  this  finding,  this  research  was 
limited  to  only  senior  level  managers.  Their  research  also 
concluded  that  there  was  no  significant  difference  in  the 
perceptions  of  the  relative  ranking  of  the  eight  factors 
between  the  organizations  of  ASD  program/project  managers 
and  AFALD  logistics  managers  except  for  the  one  factor  of 
DPML  authority  (18:67).  Based  on  this  finding,  this 
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research  was  limited  to  only  the  logistics  managers. 


The  following  "is  the  rank  order  of  the  significance  of 
the  eight  assumed  barriers  to  ILS  as  rated  by  the  ASD 
managers  and  as  rated  by  the  AFALD  managers  and  how  they 
compared"  (18:69). 

TABLE  II 

Proposed  Barriers  Ranked  by  Organization  Mean  Ratings 


ASD  ! 

AFALD 

Factor 

Mean 

Factor 

Mean 

Design  Goals 

71.73 

Design  Goals 

68.32 

Goal  Conflict 

66.05 

Goal  Conflict 

66.11 

Skills 

58.28 

DPML  Authority 

63.78 

Work  Relations 

50.65 

Skills 

58.00 

DPML  Authority 

50.40 

Logistics  T&E 

53.57 

Tools 

48.00 

Org.  Structure 

47.62 

Logistics  T&E 

47.85 

Tools 

47.14 

Org.  structure 

43.88 

Work  Relations 

42.97 

The  factors  rated  above  were  defined  on  the  survey 
instrument  in  the  following  manner: 

1.  ORGANIZATIONAL  STRUCTURE:  Primarily  the  low  relative 
position  of  the  integrated  logistics  support  office 
within  the  program  office,  and  the  dual  chain  of 
command  for  the  logistics  manager. 

2.  DPML  AUTHORITY:  The  lack  of  decision-making  authority 
delegated  to  the  logistics  manager,  such  as  inadequate 
inputs,  coordination,  or  approval  over  the  way  in  which 
program  funds  are  spent  and  other  program  decisions  are 


3.  LOGISTICS  MANAGEMENT  TOOLS;  The  misuse  or  non-use  of 
such  quantitative  and  qualitative  tools  as  Life  Cycle 
Cost  (LCC)  models.  Logistics  Supportability  Analyses 
(LSA) ,  and  Lessons  Learned  repositories  during  support 
and  product  design. 

4.  LOGISTICS  SKILLS;  Failure  to  employ  appropriately 
skilled  logisticians  during  the  different  phases  of  the 
acquisition  cycle.  Due  possibly  to  a  lack  of  skilled 
or  available  specialists. 

5.  WORKING  RELATIONS;  Lack  of  communication  or 
cooperation  between  the  logistics  personnel  and  other 
functional  specialists  within  the  program  office. 

6.  LOGISTICS  DESIGN  GOAL  DEFINITION:  Inadequate 
definition  of  logistics  design  parameters  and 
requirements  in  program  directives,  combined  with  the 
difficulty  in  translating  those  parameters  which  are 
identified  into  achievable,  verifiable  goals  for  the 
contractor . 

7.  TEST  EVALUATION:  Inadequate  T&E  for  supportability 
'  characteristics  due  to  poor  planning,  limited 

budgeting,  or  other  resource  and  time  constraints. 

8.  GOAL  CONFLICT:  For  example,  system  design  trade-offs 
which  consistently  and  forcefully  emphasize  performance 
oriented  goals  over  long-term  supportability  goals. 

The  Hull  and  Lockhart  study  also  presented  comments 

provided  by  the  subjects  to  explain  their  ratings.  These 

comments  showed  great  disparity  in  their  evaluation  of  the 

factors.  The  differences  were  often  based  on  the  phrasing 

of  the  factors,  the  wide  spectrum  of  acquisition  phases 

involved,  and  the  particular  situations  in  each  system 

program  office.  The  major  areas  they  felt  were  not 

represented  in  the  eight  factors  were  funding  control  by 

DPMLs ,  rapid  turnover  of  personnel,  lack  of  training,  and 

failure  to  become  involved  earlier  in  the  programs 

(18:103-134) . 
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Review  Conclusions 


This  literature  review  began  with  the  broad  changes  in 
the  acquisition  environment  and  continued  to  narrow  the 
focus  down  to  the  predominant  factors  that  influence  design 
changes  for  supportability .  Starting  with  the  Carlucci  and 
Thayer  programs,  the  logistics  initiative  resulted  in  the 
acquisition  regulations  and  DSARC  review  procedures  to 
reflect  the  new  emphasis  on  earlier  involvement  witn  R&M 
issues.  Other  programs,  such  as  baselining,  helped  prevent 
changes  that  bled  off  supportability  funding  for 
performance  tradeoffs.  Concluding  the  broader  issues  was 
the  study  of  how  R&M  issues  affect  the  warfighting 
capability. 

Then  several  specific  factors  were  reviewed  that 
contribute  to  increased  supportability  with  a  system's 
design.  As  a  corollary,  their  failure  to  be  fully  acted 
upon  would  be  a  constraint  to  implementing  design  changes 
for  supportability.  General  Marsh  stated  logistics 
research  and  development  needed  to  be  expanded  and  brought 
earlier  into  the  acquisition  process.  An  air  staff  officer 
expounded  on  how  vague  R&M  requirements  have  been  defined 
in  statements  of  need  and  statements  of  work.  Furthermore, 
several  articles  emphasized  the  need  for  trained  personnel 
in  the  areas  of  reliability  design  engineering  and 
acquisition  management.  The  need  for  early  testing  under 
realistic  conditions  against  viable  operational 


requirements  was  also  heavily  emphasized.  Another  area 
covert ;  was  giving  supportability  issues  equal  weight  to 
performance  in  the  evaluation  of  requests  for  proposal  and 
subsequent  contract  awards.  Similarly,  the  need  for 
incentivizing  reliability  and  maintainability  through  award 
fee  contracts  was  stressed.  Two  other  areas,  the  dual 
chain  of  command  organizational  structure  facing  the  DPML 
and  his  perceived  lack  of  authority,  were  measured  in  1982 
Hull  and  Lockhart  study.  Both  areas  were  rated  as 
significant  but  received  adverse  responses. 

While  these  R&M  issues  reflect  the  most  frequent  topics 
in  logistics  professional  journals,  they  are  not  assumed  to 
be  all-inclusive.  Factors  outside  the  managerial  realm  of 
the  Air  Force,  such  as  Congressional  influence  on  program 
stability,  were  excluded. 


III.  Research  Methodology 


Research  Strategy 

To  answer  the  four  research  questions,  a  research 
strategy  was  developed  to  collect  a  representative  sample 
of  the  perceptions  of  the  DPMLs  and  DOLs  in  Aeronautical 
Systems  Division  and  Electronic  Systems  Division  that 
manage  major  programs.  Based  on  a  literature  review  of 
what  recent  authors  and  senior  officers  felt  were  problems 
in  achieving  R&M  initiatives  and  on  the  1982  Hull  and 
Lockhart  study,  a  list  of  the  seven  most  probable 
constraints  to  initiating  design  changes  for  supportability 
was  developed.  The  list  was  then  evaluated  by  the  DPMLs  to 
measure  the  factors'  relative  impact.  Statistical  analyses 
tested  for  any  significant  differences  between  several 
different  groupings  in  how  the  logisticians  rated  the 
relative  importance  of  the  seven  factors.  In  addition,  the 
DPMLs  and  DOLs  provided  other  factors  they  perceived  as 
significant.  To  enhance  the  applied  value  of  the  research, 
their  perceptions  of  how  to  best  resolve  these  issues  were 
collected  and  presented. 

The  Instrument 


Collecting  data  of  this  type  requires  either  a  personal 
interview,  telephone  interview,  or  mailed  survey  approach. 
In  evaluating  these  three  approaches  several  factors  were 


considered.  The  mailed  survey  approach  was  rejected  due  to 
the  limited  size  of  the  population  and  the  importance  of 
attaining  a  high  response  rate.  Mail  surveys  have  been 
shown  to  have  a  strong  bias  of  nonresponse  especially  when 
the  respondents  owe  no  allegiance  to  the  sponsoring 
organization.  In  addition,  mail  surveys  would  not  have 
been  suitable  for  attaining  large  amounts  of  information  or 
probing  deeply  into  questions  as  this  research  required. 
Likewise,  respondents  tend  to  refuse  to  cooperate  when  the 
mail  questionnaire  is  long  and  complex  as  this  research 
would  have  required  (10:308).  The  telephone  interview 
approach  was  rejected  since  that  instrument  prohibits  the 
use  of  complex  scales  and  questioning  which  is  inherent  in 
evaluating  the  relative  significance  of  7  complex 
abstractions  (10:307). 

The  personal  interview  approach  was  selected  as  most 
appropriate  for  this  research  effort  since  it  eliminates 
the  most  serious  shortcomings  of  the  alternatives.  In 
addition,  the  personal  interview  provided  three  additional 
advantages : 

1)  It  would  encourage  greater  depth  and  detail  of 
information  since  the  subject  was  more  likely  to 
concentrate  and  devote  time  to  the  question. 

2)  The  volume  and  quality  of  their  proposed  solutions 
would  increase  since  it  is  easier  for  respondents 
to  vocalize  opinions  than  to  write  them. 


Similarly,  the  personal  interview  would  allow 
expansion  on  proposed  solutions  and  through  the 
interaction,  the  researcher  could  gain  a  more 
complete  understanding  of  the  subjects  comments 
and  physical  cues  (18:33-34). 

3)  There  would  be  an  improved  quality  of  responses  by 
enabling  the  interviewer  to  monitor  the  conditions 
of  the  interview,  prescreen  respondents,  and  to 
adjust  the  language  of  questions  by  providing 
on-the-spot  explanation  in  order  to  limit  missing 
and  invalid  data  (10:294) . 

A  structured  interview  approach  was  selected  and 
designed  to  be  self-explanatory  so  that  it  could  stand 
alone  or  with  little  explanation.  The  structured  approach 
was  selected  for  the  following  reasons: 

1)  Increased  uniformity  between  interviews  through 
standardized  wording. 

2)  Minimized  diversity  of  interpretation  of  the 
questions . 

3)  Increased  reliability  in  comparing  results  by 
maintaining  a  similar  instrument  to  the  Hull  and 
Lockhart  study. 

Attachment  A  is  a  copy  of  the  interview  form  used  and 
is  crucial  in  understanding  the  remainder  of  this  study. 

The  five  page  form  consisted  of  five  parts:  the  survey 
information  sheet  with  a  hypothetical  scenario  and  set  of 


instructions,  a  personal  data  sheet  to  collect  demographic 
information  on  the  respondents,  a  listing  of  the  seven 
factors  being  rated  with  a  short  definition  of  each,  a 
graphic  rating  scale  to  rate  the  seven  factors,  and  a  sheet 
containing  two  open  ended  questions  for  gathering 
additional  constraints  and  recommended  solutions  to  all  the 
constraints . 

The  hypothetical  scenario  and  instructions  helped 
foster  uniformity.  The  hypothetical  case  was  intended  to 
encourage  the  respondents  to  use  both  their  past 
experiences  and  personal  judgement.  The  instructions  were 
designed  to  help  facilitate  the  inherent  complexity  of 
marking  the  ratings  of  the  seven  factors  on  a  graphic 
scale.  The  instructions  encouraged  the  reading  of  all  the 
factors  before  attempting  to  rate  them.  This  was  done  in 
order  to  prevent  the  position  of  the  factors  on  the  list, 
from  affecting  the  rating  of  the  factors  (18:34-35).  The 
five  demographic  questions  on  the  personal  data  sheet 
enabled  statistical  analyses  of  the  respondents'  ratings 
by  various  experience  level  measurements  and  by  product 
divisions.  The  seven  factors  rated  were  selected  from  the 
results  of  the  literature  review.  The  factors  were 
randomly  arranged  in  order  not  to  influence  their  ratings. 
The  format  and  size  of  their  definitions  were  also  kept 
compatible.  The  definitions  included  examples  of  certain 
issues  that  would  be  considered  part  of  that  factor.  Every 
attempt  was  made  to  avoid  any  biasing  through  terminology. 


t 

I 

The  graphic  scale  was  selected  due  to  its  ability  to 
generate  both  the  rank-ordered  data  to  respond  to  research 
question  one  and  the  interval  data  for  the  statistical 
tests  to  respond  to  question  two  (18:35).  Additionally, 
the  scale  was  used  in  the  Hull  and  Lockhart  study  and  its 
continuance  would  enhance  the  reliability  of  comparing  the 
results  of  the  two  studies.  The  basic  assumption  was  that 
|  the  DPMLs  could  and  would  make  good  judgements  (10:263). 

Since  all  the  factors  were  rank  ordered  on  the  same  scale, 
the  respondents  were  visually  cued  to  interpret  their 
perceptions  in  terms  of  the  same  standard.  With  each 
factor  placed  relative  to  the  others,  a  rank  order  was 
implied  (18:35-36)  . 

Several  guidelines  were  followed  in  developing  the 
graphic  scale  (18:36). 

1)  The  rating  line  was  long  enough  to  allow 
discrimination  among  the  seven  factors,  but  not 
long  enough  to  disrupt  the  rater's  unity  of 
continuum . 

2)  The  line  was  continuous  depicting  the  continuity 
of  the  effects  on  R&M  being  measured. 

3)  The  "high"  impact  end  was  placed  at  the  top. 

4)  To  discourage  error  of  central  tendency,  the 
phrasing  of  the  descriptive  cues  on  the  ends  were 


not  too  extreme. 


5)  The  end  cues  were  set  in  enough  to  allow  room  for 


more  extreme  ratings. 

6)  The  rankings  were  measured  in  millimeters  thus 
enabling  numerical  values  to  be  assigned. 

The  fourth  part  consisted  of  two  open  ended  questions. 
The  first  requested  any  additional  constraints  that  were 
not  included  in  the  seven  factors  in  order  to  insure  any 
important  constraints  were  not  overlooked.  The  second 
elicited  what  the  DPMLs  believed  to  be  the  best  course  of 
action  to  reduce  the  impact  of  the  various  constraints. 

Instrument  Pretest 

The  instrument  was  pretested  on  the  Air  Force  Institute 
of  Technology  personnel.  Due  to  the  similarity  to  the  Hull 
and  Lockhart  instrument,  the  only  objectives  were  to  gain 
experience  in  administering  the  interview,  to  determine  if 
the  interview  time  was  reasonable,  to  evaluate  the  clarity 
of  the  test,  and  to  evaluate  content  validity  (18:37).  As 
a  result,  a  few  changes  were  made  in  the  wording  for 
clarification  and  some  administrative  techniques  were 
modified . 

Sample  Selection 

The  target  population  was  determined  to  be  the  current 
DPMLs  and  DOLs  assigned  to  major  acquisition  programs 
within  Aeronautical  Systems  Division  and  Electronic  Systems 
Division.  Experience  was  important  for  two  reasons  in  this 


study.  First,  the  ratings  of  the  seven  listed  factors  by 
inexperienced  personnel  could  skew  the  statistical  results. 
Second,  qualitative  answers  to  research  question  four, 
solutions  to  reducing  the  impact  of  the  constraints, 
required  a  thorough  knowledge  of  what  happens  in  the  early 
acquisition  phases  and  how  any  changes  might  affect  other 
related  activities. 

However,  due  to  the  very  small  number  of  DPMLs  and  DOLs 
that  had  even  one  year  experience  as  a  DPML  in  the  early 
acquisition  phases,  the  target  population  was  looked  at  as 
three  concentric  circles.  First,  28  of  the  32  major  program 
DPMLs  and  DOLs  at  ASD  and  ESD  were  interviewed  and 
statistical  results  were  calculated.  Then  statistical 
analysis  were  recalculated  on  a  second  group  containing  only 
the  15  individuals  that  had  three  or  more  total  years  within 
acquisition  and  1  year  as  a  DPML  during  any  phase.  Finally, 
the  statistical  analysis  was  again  recalculated  for  a  third 
group  containing  the  10  people  who  had  3  or  more  years  total 
acquisition  experience,  but  also  1  year  as  a  DPML  in  the 
concept  exploration,  demonstration  validation,  or  early 
portions  of  full  scale  development  phase.  The  assumption 
was  made  that  an  incumbent  of  a  DPML  position  in  a  major 
program  for  these  periods  of  time  was  relatively 
knowledgeable.  No  attempt  was  made  to  differentiate  based 
on  personal  factors. 


Having  defined  the  target  populations,  the  first  step 
was  determining  the  major  programs  within  each  of  the  two 


product  divisions  and  their  incumbents.  Senior  staff 
members  in  the  Air  Force  Acquisition  Logistics  Center 
provided  me  an  alpha  roster  of  these  positions,  their 
incumbents,  the  current  work  locations  and  phone  numbers. 
Seventeen  positions  were  identified  within  ASD  and  16  of 
them  were  interviewed.  Fifteen  were  identified  within 
ESD.  Due  to  the  travel  restrictions  and  scheduling 
problems  only  12  of  these  were  interviewed. 

Subject  Contact  Plan 

Because  of  the  travel  requirement  to  ESD,  it  was 
determined  the  optimum  time  for  these  interviews  was  during 
a  break  in  the  researcher's  classes.  Therefore,  all  ESD 
DPMLs  were  contacted  and  interviewed  first,  and  then  the 
ASD  subjects  were  contacted  later  over  the  next  month.  In 
both  divisions,  the  subjects  were  contacted  by  telephone, 
the  researcher  introduced  himself,  explained  the  research 
objectives  and  time  requirements,  and  established  an 
appointment.  A  record  was  maintained  of  all  contacts  and 
interviews.  These  were  correlated  to  a  numbered  interview 
schedule  which  enabled  follow-up  contact  but  also  provided 
anonymity  for  the  subjects. 


Conducting  the  Interview 

In  order  to  improve  subject  receptiveness,  each 
interview  was  preceded  by  brief  introductions,  explanations 
of  the  objectives  of  the  study,  the  beneficial  values  of 
the  findings,  and  a  reassurance  of  the  confidentiality  of 
the  interview.  The  subject  first  completed  the  demographic 
questions.  The  subject  then  received  the  instruction  sheet 
and  the  graphic  scale  for  rating  the  seven  factors.  Though 
designed  to  stand  alone,  if  the  subject  had  any  interpre¬ 
tation  questions  they  were  answered.  However,  interaction 
at  this  point  was  purposely  limited  in  order  to  reduce  any 
bias  on  the  rating  results  (18:42-43). 

After  finishing  the  rating  exercise,  the  subject  was 
then  asked  to  identify  any  other  significant  constraints 
not  listed  or  covered  by  the  seven  factors.  Finally,  the 
subject  was  encouraged  to  propose  any  recommended  solutions 
for  reducing  the  impact  of  any  of  the  constraints.  This 
portion  was  undoubtedly  the  most  difficult  to  administer  in 
that  it  required  motivating  the  DPMLs  to  be  innovative  and 
as  specific  as  possible  in  their  recommendations.  The 
researcher  recorded  their  comments  (usually  paraphrased)  on 
the  numbered  raw  data  collection  sheets. 

Data  Analysis 

The  research  objectives  identified  four  goals  for 
analyzing  the  interview  data.  The  first  goal  was  to 


determine  how  the  DPMLs  rank-order  certain  constraints  that 


impair  their  ability,  during  the  early  acquisition  phases. 


ft 


to  influence  a  system's  design  for  improved  supportability . 
To  do  this,  individual  ratings  for  each  factor  were 
quantified  by  measuring  the  distance  in  millimeters  from  the 
bottom  of  the  scale  to  the  respective  hash  mark  rating  by 
the  subject.  Appendix  B  is  the  responses  of  the  three 
groups  that  were  input  into  the  computer.  The  SPSS 
statistics  package  then  calculated  the  group  means  from  the 
individual  scores.  Appendix  C  is  the  Statistical  Package 
for  the  Social  Sciences  (SPSS)  program  code  used  to  generate 
the  statistical  information.  The  factors  were  then  listed 
by  magnitude  of  the  mean  ratings.  Though  the  list  was  not 
considered  all  inclusive  or  absolute,  it  did  provide  a 
prioritized  listing  of  the  major  program  DPMLs  perceived  the 
relative  significance  of  these  factors. 

The  second  objective  was  to  determine  if  the  DPMLs 
significantly  differed  in  their  perceptions  by  their  product 
divisions.  As  stated  earlier,  statistical  analyses  were 
repeated  after  omitting  first  the  less  experienced  in 
overall  acquisition  and  then  again  after  omitting  those 
having  little  experience  in  the  early  acquisition  phases. 
T-tests  were  based  on  the  assumption  that  the  underlying 
population  distributions  were  normal  and  the  variances  were 
near-equivalent.  Previous  research  has  shown  that  most 
opinion  surveys  approximate  a  normal  distribution.  However, 
even  when  the  distribution  differs,  the  statistical  tests 
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are  relatively  unaffected  (18:45-46).  The  t-test  procedure 
also  analyzed  the  differences  in  the  group  means  between 
the  two  product  divisions  based  on  the  three  target 
groupings.  The  mean  score  differences  for  each  of  the 
seven  factors  was  tested  at  the  .05  significance  level. 

Due  to  the  length  of  the  output  data,  samples  of  the 
pertinent  tests  are  included.  Appendix  D  is  a  sample  of 
the  computer  output  data  comparing  the  mean  ratings  of  ASD 
and  ESD  and  the  statistical  tests  performed  by  the 
researcher  on  the  computer  information.  In  addition,  the 
three  target  groups  were  also  tested  for  significant 
differences  based  on  grade,  time  in  acquisition,  time  as  a 
DPML ,  and  time  in  the  early  phases.  Appendices  E  and  F 
contain  samples  of  these  statistical  tests. 

The  third  objective  was  to  identify  any  other  factors 
the  DPMLs  felt  were  significant.  The  raw  data  collection 
sheets  were  read,  evaluated,  and  then  placed  into  natural 
groupings.  Though  considerable  subjectivity  was  involved 
in  this  analytical  method,  every  attempt  was  made  to  remain 
objective . 

The  fourth  objective  was  to  organize  and  present  the 
respondents'  perceptions  of  how  best  to  resolve  these 
identified  constraints.  Though  they  agreed  as  to  what  they 
believed  were  the  most  and  least  significant  constraints, 
they  often  varied  in  what  they  felt  should  be  done,  if 
anything,  to  resolve  the  constraints.  After  each  factor. 
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the  various  proposed  solutions  were  listed  in  order  of  the 
solutions  mentioned  most  often.  When  only  a  few  mentioned 
a  solution,  it  is  not  inferred  that  others  agreed  or 
disagreed  for  they  merely  may  not  have  thought  of  that 
solution  at  the  time.  No  attempt  is  made  to  judge  which 


would  be  the  best. 
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IV.  Findings  and  Analysis 


Introduction 

In  ASD  and  ESD  there  are  32  designated  DPMLs  and  DOLs  of 
major  system  programs.  Twenty  eight  of  these  people  were 
interviewed  and  their  responses  make  up  the  data  base  for 
this  research.  When  the  research  was  originally  planned,  it 
was  assumed  that  the  DPMLs  of  these  largest  programs  had 
extensive  acquisition  and  DPML  experience.  In  addition,  it 
was  assumed  that  their  experience  levels  were  somewhat 
comparable.  Once  the  wide  variance  in  experience  levels 
were  discovered,  the  28  respondents  were  categorized  into 
three  concentric  groups. 

The  first  group  included  the  entire  28  respondents.  The 
second  group  included  only  the  15  people  in  group  one  who 
had  three  or  more  years  total  acquisition  experience  and  at 
least  one  year  as  a  DPML  during  any  phase.  The  third  group 
included  only  the  ten  people  in  the  second  group  that  had 
one  year  experience  as  a  DPML  in  the  early  phases  of  concept 
exploration,  demonstration  validation,  or  early  portions  of 
full  scale  development.  By  dividing  the  respondents  into 
these  groups,  it  was  possible  to  determine  if  the 
differences  in  their  experience  levels  was  of  any 
significance.  These  findings  are  presented  during  the 
analysis  of  research  question  two.  The  demographic 
composition  of  the  three  groups  are  summarized  in  Table  III. 
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TABLE  III 


Demographic  Composition  of  Respondents 


Grouped  By: 

Group  1 

Group  2 

Group  3 

Product  Division: 

ASD 

16 

10 

6 

ESD 

12 

5 

4 

Grade : 

05/GS14  and  below 

23 

10 

7 

06/GS15 

5 

5 

3 

Experience  in  Early  Phases: 

Less  than  1  year 

16 

5 

— 

One  or  more  years 

12 

10 

10 

Overall  Acquisition  Experience: 

Two  or  less  years 

8 

— 

— 

Three  or  more  years 

20 

15 

10 

DPML  Experience: 

Less  than  1  year 

11 

— 

— 

One  or  more  years 

17 

15 

10 

Research  Question  One 

The  first  objective  was  to  identify  how  the  DPMLs 
rank-order  the  seven  constraints  as  to  their  impact  on 
impairing  the  DPML's  ability  to  influence  a  system's  design 
for  improved  supportability .  To  satisfy  this  objective,  the 
factors  were  ranked  according  to  each  factor's  mean  score  or 
average  measurement  on  the  graphic  rating  scale.  The 
rankings  in  Table  IV  show  how  the  DPMLs  and  DOLs  generally 
perceived  the  constraints. 


TABLE  IV 

Proposed  Constraints  as  Ranked  by  All  DPMLs  and  DOLs 


Rank 


Factors 

Mean 

Requirements  Definition 

65.5 

Trained  Personnel 

59.8 

RFP  Evaluation  Criteria 

51.3 

Logistics  R&D 

43.4 

Test  and  Evaluation 

35.3 

DPML  Authority 

34.6 

Organizational  Structure 

21.6 

The  means  of  the  ratings  in  Table  IV  are  spread  out 
leaving  a  clear  distinction  between  the  ranking  of  the 
factors  with  the  only  exception  being  between  Test  and 
Evaluation  and  DPML  Authority  which  are  both  rated  low  in 
fifth  and  sixth  place.  However,  since  the  total  group  of 
respondents  had  less  DPML  experience  and  early  acquisition 
experience  than  originally  expected,  it  was  prudent  to  also 
rank  the  means  of  the  factors  as  perceived  by  group  two  and 
group  three  to  see  if  there  were  any  significant 
differences.  The  ranking  and  mean  ratings  of  group  two  and 
group  three  are  shown  in  Table  V. 


TABLE  V 

Proposed  Constraints  as  Ranked  by  the  More 
Experienced  DPMLs  and  DOLs 


As  can  be  seen  by  comparing  the  rankings,  all  three 
groups  agreed  on  the  ranking  of  the  seven  factors.  The  F- 
tests  and  t-tests  performed  on  each  of  the  factors  rated 
support  these  rankings.  This  implies  that  the  DPMLs  and 
DOLs  of  the  major  programs,  regardless  of  experience  level 
have  a  generally  uniform  perception  of  the  impact  of  the 
seven  factors  on  impairing  their  ability  to  influence  a 
system's  design  for  improved  supportability .  None  of  the 
differences  in  the  rated  means  were  found  to  be 
statistically  significant  at  the  a=.05  significance  level. 
Though  not  significant  differences,  there  were  a  few 
interesting  trends  among  the  various  experience  levels. 


Group  two  rated  the  factors  Trained  Personnel  and  Test  and 
Evaluation  higher  than  group  one,  while  group  three  rated 
those  same  factors  even  higher  than  group  two. 

Conversely , the  more  experienced  respondents  placed  less 
emphasis  on  the  factor  of  DPML  Authority.  Group  two  rated 
it  lower  than  group  one  and  group  three  rated  it  even  lower 

Research  Question  Two 

The  second  objective  was  to  identify  if  the  DPMLs  in 
ASD  and  ESD  significantly  differed  in  their  perception  of 
the  rank  ordering  of  the  seven  constraints.  Using  a  t-test 
at  a  significance  level  of  a=.05,  the  two  product  divisions 
were  not  significantly  different  on  any  of  the  seven 
factors.  In  all  but  the  Requirements  Definition  factor, 
the  variances  were  found  to  be  equal,  so  the  pooled 
variance  estimator  was  used.  In  the  Requirements 
Definition  factor  a  separate  variance  estimator  was  used. 
Table  VI  shows  the  comparison  of  rankings  in  ASD  and  ESD. 


TABLE  VI 

Comparative  Rankings  of  all  ASD  and  ESD  Respondents 


ASD 


Rank  Factor 


Requirements  Def. 
Trained  Personnel 
RFP  Eval.  Criteria 
Logistics  R&D 

5  Test  and  Eval. 

6  DPML  Authority 
Org.  Structure 


ESD 

Factor 

Mean 

Requirements  Def. 

70 

Trained  Personnel 

59 

RFP  Eval.  Criteria 

51 

Logistics  R&D 

39 

Test  and  Eval. 

37 

DPML  Authority 

29 

Org.  Structure 

T9 

The  factor  generating  the  widest  range  of  opinions,  or 
largest  standard  deviation,  within  both  ASD  and  ESD  was  the 
perceptions  of  the  impact  of  DPML  Authority.  Interestingly 
enough,  this  same  factor  was  the  only  factor  in  the  Hull 
and  Lockhart  study  on  which  the  ASD  and  AFALD  respondents 
differed  significantly. 

ASD/ESD  Experienced  DPMLs  Compared.  To  insure  that  the 
more  experienced  DPMLs  and  DOLs  (groups  two  and  three)  in 
ASD  and  ESD  did  not  differ  significantly,  they  were  also 
tested.  The  rankings  of  group  two  and  group  three  differed 
from  each  other  slightly  and  both  differed  from  group  one. 
However,  none  of  the  differences  were  statistically 


significant  at  the  a=.05  level.  Though  not  statistically 
significant,  the  biggest  difference  in  comparing  the  more 
experienced  respondents  within  ASD  and  ESD  was  those  in  ASD 
ranked  Logistics  R&D  third  with  a  mean  of  53  while  ESD's 
experienced  respondents  ranked  it  fifth  with  a  mean  of  only 
27.  An  explanation  for  this  difference  might  be  that  ASD 
had  more  Colonel  DOL  respondents  who  all  rated  Logistics 
R&D  higher  than  did  the  junior  DPMLs . 

Junior/Senior  Respondents  Compared.  In  addition  to 
comparing  ASD  and  ESD  respondents,  the  perceptions  of  the  • 
junior  and  senior  grade  respondents  were  also  compared.  In 
all  of  the  various  comparisons  done  in  this  study,  the 
largest  variance  of  answers  within  each  group  occurred  when 
grouped  by  grade.  In  addition  to  the  pooled  varianced 
t-tests,  which  showed  no  statistically  significant 
difference  at  the  a=.05  level,  the  large  standard 
deviations  within  each  group  showed  there  was  not  any  real 
consistency  based  on  rank.  The  only  general  tendency  was 
that  in  all  three  experience  level  groups,  the  senior 
respondents  ranked  Logistics  R&D  higher  and  RFP  Evaluation 
Criteria  lower  than  the  junior  respondents. 

Inexperienced/Experienced  Respondents  Compared.  A 
third  comparison  was  made  between  the  respondents  with  less 
than  three  years  of  acquisition  experience  with  those  with 
three  or  more  years  experience  in  Table  VII.  When  this 
comparison  was  done,  one  factor  was  found  to  be 


statistically  significant  at  the  a=.05  level.  The 
inexperienced  managers  rated  the  factor  of  DPML  Authority 
significantly  higher  than  the  experienced  respondents. 

This  difference  in  perceptions  seems  logical  since  the 
newer  personnel  often  lack  many  of  the  skills  and  rank  that 
enable  the  more  experienced  people  to  work  around  the  lack 
of  a  formal  power  base. 

Another  factor  was  almost  statistically  significant  at 
the  a=0.5  level.  The  Test  and  Evaluation  factor  was  rated 
35%  lower  by  the  less  experienced  group  than  by  those  with 
three  or  more  years  in  acquisition.  Several  inexperienced 
respondents  stated  this  is  an  area  they  were  unfamiliar 
with  and  thus  rated  it  lower  for  that  reason. 


TABLE  VII 

Comparison  of  Inexperienced  and  Experienced  Respondents 

in  Acquisition 
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Time  as  a  DPML  Compared.  A  fourth  comparison  was  made 
between  the  respondents  with  less  than  one  year  as  a  DPML 
and  those  with  one  or  more  years  experience.  None  of  the 
differences  in  factor  ratings  were  statistical ly 
significant  at  the  a=.05  level.  Even  though  DPML  authority 
was  again  rated  higher  by  the  inexperienced  DPMLs,  it  was 
much  less  pronounced  since  many  of  the  newer  DPMLs  had 
spent  several  years  in  logistics  acquisition  before 
becoming  a  DPML. 

Early/Late  Acquisition  Experience  Compared.  A  fifth 
comparison  was  made  between  the  respondents  with  less  than 
one  year  in  the  early  acquisition  phases  and  those  with  one 
or  more  years  experience.  None  of  the  differences  in 
factor  ratings  were  statistically  significant  at  the  a=.05 
level.  The  less  experienced  respondents  again  rated  DPML 
Authority  higher.  In  addition,  the  experienced  respondents 
also  rated  Trained  Personnel  almost  significantly  higher. 

Research  Question  Three 

The  third  objective  was  to  identify  any  other 
significant  constraints  that  impact  the  DPMLs.  Fourteen  of 
the  28  respondents  offered  other  significant  constraints 
which  were  subjectively  grouped  into  seven  areas.  Though 
some  were  closely  related  to  the  original  seven 
constraints,  all  responses  were  listed.  The  number  of 
respondents  who  mentioned  each  issue  is  given  to  only 
provide  a  better  perspective  of  the  overall  responses . 


58 


When  only  a  few  mentioned  a  constraint,  it  is  not  inferred 
that  others  agreed  to  disagreed  for  they  merely  may  not 
have  thought  of  it  at  the  time.  Likewise,  this  should  not 
be  considered  an  inclusive  list  of  all  additional 
constraints  to  the  original  seven.  However,  it  does 
provide  a  good  "snapshot"  view  of  what  additional 
constraints  were  on  the  minds  of  the  DPMLs  on  the  days  they 
were  interviewed. 

1.  Program  Director  (PD)  priorities.  Nine 
respondents  felt  if  a  PD  is  not  convinced  that  R&M 
expenditures  are  of  a  high  priority,  none  of  the  DPML's 
initiatives  will  be  successful.  They  stated  that  SPOs  only 
contract  and  incentivize  what  the  PD  wants  since  it  is  AFSC 
not  AFLC  money  being  used. 

2.  Manning  Issues.  Several  respondents  brought  up 
manning  constraints  that  fell  within  six  areas: 

2.1  Manpower  Authorizations.  Eight  respondents 
remarked  there  is  no  overall  manning  plan  or  OPR  since  the 
logistics  personnel  come  from  6  or  more  military  and 
civilian  AFSCs.  Currently  the  slots  are  justified  through 
various  channels,  and  the  SPOs  applying  the  most  pressure 
get  the  most  people.  Almost  all  of  the  respondents  felt 
there  were  not  enough  people  assigned  (trained  or 
untrained)  to  do  the  job  right. 

2.2  Continuity.  Six  criticized  the  "revolving 
door"  of  personnel  turnovers  which  allowed  the  experienced 


personnel  to  leave  without  regard  to  their  criticality  in  a 
program  or  its  timing  to  significant  milestones. 

2.3  Relation  to  R&M  Engineers.  Six  also  remarked 
that  having  the  R&M  engineers  work  in  the  engineering 
section  instead  of  for  the  DPML  creates  problems  in 
establishing  consolidated  logistic  efforts. 

2.4  Basket  SPOs .  Three  stated  that  the  AFALC 
staff  does  not  understand  that  the  smaller  SPOs  do  not  have 
the  resources  to  train  their  own  people  on  the  job  as  well 
as  the  larger  SPOs.  The  AFALC  often  assigns  the  new 
trainees  to  the  smaller  SPOs  and  then  moves  them  to  the 
bigger  SPOs  after  they  become  productive. 

2.5  ALC  Support.  Two  brought  up  the  point  that 
the  ALCs  are  too  reluctant  to  commit  travel  funds  and 
personnel  during  the  early  phases  of  a  program.  They  felt 
this  leads  to  important  sources  of  expertise  not  being 
available  during  the  times  they  could  have  the  biggest 
effect. 

2.6  Meaningful  Experience.  One  felt  that 
designing  for  maintainability  requires  common  sense  and 
knowledge  of  the  field  being  supported.  He  felt  too  many 
inexperienced  people  have  never  worked  in  the  field  or 
understood  the  human  factors  involved. 

3.  Inability  to  Acquire  Assistance.  Several  mentioned 
that  the  lack  of  real  assistance  from  the  AFALC  staff 
prevented  the  development  of  more  R&M  initiatives  since  it 
took  the  DPMLs '  staffs  longer  to  become  proficient. 
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4.  Funds  Control.  Three  stated  that  the  DPMLs ' 
failure  to  understand  the  funding  process,  tracking  of 
funds,  or  availability  of  depot  funds  has  caused  many 
initiatives  to  be  dropped  needlessly  for  lack  of  funds. 
Likewise,  they  felt  the  DPMLs  failed  to  properly  coordinate 
with  AFLC ,  AFSC ,  Air  Staff,  and  other  organizations  for 
initial  spares  funding  and  other  available  sources. 

5.  Contractor  Issues.  Two  contractor  related 
constraints  were: 

5.1  Organization  Structure:  Five  suggested  that 
unless  a  contractor's  organization  is  conducive  to  the 
achievement  of  R&M  objectives,  the  program  is  much  harder 
to  manage. 

5.2  Failure  to  Challenge.  Two  mentioned  that 
DPMLs  are  often  awed  by  contractors  which  becomes  a 
psychological  constraint.  Because  contractors  wear  a  coat 
and  tie,  the  military  personnel  often  think  the  contractor 
knows  more  and  accept  things  without  challenging  them. 

6.  Policy  and  Guidance  Issues.  Three  policy  related 
constraints  were: 

6.1  Conflicting  Guidance.  Three  felt  the 
regulations  and  directives  are  ambiguous.  In  addition,  the 
headquarters  guidance  from  services,  commands,  and  product 
divisions  often  conflict. 

6.2  Sole  Source.  Two  felt  slow  approval 
procedures  for  sole  sourcing  hampers  and  prevents  many  R&M 
initiatives  in  contracts. 


Research  Question  Four 

The  fourth  objective  was  to  organize  and  present  the 
respondents'  perceptions  of  how  to  best  resolve  these 
identified  constraints.  First,  the  solutions  to  the  seven 
ranked  constraints  will  be  presented  in  the  order  they  are 
listed  on  the  interview  sheet.  Then  the  solutions  to  the 
constraints  identified  in  research  question  three  will  be 
presented.  The  statements  below  are  not  exact  quotations 
of  the  respondents  but  are  summarized  and  paraphrased  as 
closely  as  possible  by  the  author. 


Requirements  Definition 

In  the  rating  exercise,  the  requirements  definition 
factor  was  the  most  consistently  ranked  factor  as  evidenced 
by  the  lowest  standard  deviations  in  the  exercise.  Though 
the  majority  of  the  respondents  rated  it  as  the  most 
important  constraint,  they  all  agreed  that  in  the  last  five 
years  the  Air  Force  has  improved  tremendously  in  defining 
the  requirements.  In  fact,  some  of  the  respondents  that  did 


not  offer  any  solutions  said,  "we  are  there".  One  pointed 
out  that  in  his  program,  the  requirements  definition  of 
manpower  constraints  also  translated  to  programmed  manpower 
reductions.  However,  the  fear  expressed  by  many  was  that 
R&M  is  only  in  vogue  now  and  we  may  not  support  it  with  the 
same  consistency  in  three  or  four  years.  Seventeen 
respondents  suggested  solutions  that  fell  into  three 
categories:  the  use  of  user  conferences,  earlier  use  of 


baselining  logistics  issues,  and  the  development  of 
logistics  analysis  tools. 

1.  User  Conference.  Ten  respondents  suggested  the 
Air  Force  could  improve  defining  the  requirements  by 
establishing  an  early  working  conference  with  the  user  in 
order  to  discuss  some  important  issues  before  too  many 
critical  decisions  are  made.  Most  felt  the  user  is  really 
not  in  a  good  position  to  develop  the  SONS.  First,  they 
are  naturally  reluctant  to  commit  themselves  to  certain 
stated  needs  until  they  know  what  is  available  with  respect 
to  technology  and  other  new  concepts.  Second,  since  the 
users  are  not  continually  involved  in  planning  new  Air 
Force  designs,  inexperienced  people  often  unintentionally 
limit,  in  the  early  system  documentation,  the  best 
intentions  of  later  logisticians.  Past  statements  of  need 
(SONs)  have  grossly  underplanned  for  the  logistical 
support.  Newer  concepts  such  as  line  replaceable  modules, 
time  stress  measurement  devices,  and  systems  totally 
integrated  to  one  another  must  be  planned  for  from  the  very 
beginning.  The  user  conference  would  give  logisticians  in 
the  user  command  a  forum  to  receive  this  guidance  in  the 
early  requirements  process. 

Most  felt  the  best  timing  for  this  conference  would  be 
at  the  end  of  the  concept  exploration  study  effort  but 
before  going  to  Air  Staff  for  a  Program  Management 
Decision.  At  that  time  a  technical  staff  made  up  of  R&M 


engineers,  logisticians,  and  contractors  from  the  product 
division,  pertinent  laboratories,  and  the  likely  Air 
Logistics  center  would  meet  with  the  user.  This  more 
experienced  group  could  then  revise  the  statement  of  need 
for  a  more  realistic  approach  to  support.  In  addition, 
they  could  outline  a  proposed  maintenance  concept  to 
include  levels  of  maintenance,  mean  time  between  failures 
(MTBFs ) ,  mean  times  to  repair  (MTTRS) ,  deployment  support 
concepts,  and  other  important  areas.  These  could  then  be 
transferred  later  into  meaningful  R&M  factors  for  designers 
to  work  toward.  Throughout  this  period,  it  would  be 
imperative  for  the  logisticians  to  work  very  closely  with 
the  engineers.  AFLC  would  have  to  address  what  new 
procedures  would  be  required  as  well  as  any  additional 
funding  and  manning  requirements.  Though  this  process 
could  be  expensive,  all  ten  of  the  respondents  felt  that 
this  early  commitment  would  do  more  for  the  logistics 
effort  than  larger  SPO  staffs  in  later  phases  trying  to 
work  around  the  original  problems. 

Several  of  the  respondents  stated  the  current 
definitions  are  not  stringent  enough  and  this  early 
interface  could  define  more  realistic  reliability  rates 
that  the  SPO  could  provide.  Likewise,  the  conference  could 
provide  the  users  an  introduction  to  new  logistics 
technology  they  may  not  have  been  aware  of,  yet  prevent  the 
quest  for  items  the  current  technology  could  not  support. 


Two  respondents  pointed  out  that  most  system  ideas  are  sold 
to  MAJCOMS  by  industry  representatives.  These  conferences 
could  show  the  users  other  alternatives  and  emphasize  to 
industries  that  all  program  concepts  will  be  competed.  In 
addition,  the  conference  would  insure  close  coordination 
with  the  PEMs  for  reasonable  financial  limits.  This  would 
help  to  eliminate  working  various  cost  options  that  would 
not  be  politically  feasible. 

2.  Baselining.  Four  respondents  felt  the  best 
solution  was  through  establishing  the  logistics 
requirements  in  the  early  RFPs  and  not  allowing  them  to  be 
changed  without  extraordinary  upper  level  coordination. 
Likewise,  they  felt  the  traditional  baselining  of  the 
programs  should  occur  earlier,  preferably  right  after  Full 
Scale  Development  source  selection.  Similarly,  one 
respondent  felt  the  best  solution  was  to  stand  tougher  on 
the  requirements.  He  recommended  "do  or  die"  tests  that 
will  not  accept  systems  that  do  not  meet  requirements. 

3.  Tool  Development.  Three  respondents  stated 
requirements  could  be  defined  more  clearly  if  logisticians 
could  develop  tools  to  determine  what  reliability 
requirements  (ie.  95%  or  98%)  were  really  needed. 

Currently  logisticians  are  unable  to  state  how  many  more 
sorties  or  other  operational  goals  could  be  achieved  with  a 
2%  increase  in  reliability  of  a  certain  system.  DPMLs 
should  be  able  to  quantify  R&M  impacts  in  terms  of 


dollars.  These  tools  would  also  be  valuable  in  determining 
support  required,  spares,  and  life  cycle  costs.  These 
specific  figures  would  enable  DPMLs  to  have  a  much  stronger 
bargaining  position  in  tradeoff  discussions  of  cost  versus 
requirements.  Though  Dynametrics  and  other  simulation  models 
are  a  start  in  that  direction,  academia  and  staff  specialists 
should  cooperate  to  develop  these  management  tools. 

RFP  Evaluation  Criteria.  The  majority  of  the 
respondents  again  believed  the  Air  Force  has  made 
remarkable  improvements  in  this  area  during  the  last  five 
years.  In  addition,  most  felt  the  requirement  definition 
problem  was  very  closely  related  to  evaluating  the  RFPs. 
Therefore,  many  of  the  recommended  solutions  could  be 
integrated  or  combined.  Seventeen  respondents  suggested 
solutions  which  were  grouped  into  three  areas:  use  of  a 
source  selections  cadre,  development  of  better  RFPs  to  the 
contractor,  and  continued  emphasis  on  the  program  directors 
to  be  responsive  to  the  logistic  issues. 

1.  Source  Selection  Cadre.  Six  respondents 
suggested  that  because  of  several  existing  conditions,  the 
best  way  to  evaluate  logistics  issues  in  RFPs  would  be  to 
have  a  trained  cadre  of  logisticians.  Some  of  the  points 
they  feel  justify  this  recommendation  are  as  follows. 

First,  the  RFP  for  full  scale  development  occurs  when  the 
SPO's  logistics  staff  is  very  small  and  does  not  have 
sufficient  people  to  evaluate  all  the  areas  even  if  they 


were  fully  capable.  Second,  the  current  system  tends  to 
prevent  experienced  source  selection  personnel  from 
participating  a  second  time  when  they  could  provide 
valuable  guidance.  Similarly,  since  this  selection  process 
often  takes  up  to  six  months,  when  other  SPOs  are  tasked  to 
provide  "bodies",  there  is  a  tendency  to  send  those  who  are 
marginally  productive.  Third,  source  selection  requires 
many  specific  skills  that  could  be  better  developed  and 
then  used  more  efficiently  on  a  selection  cadre  team. 
Specific  skills  necessary  for  the  logistics  people 
include:  the  working  intricacies  of  Logistics  Support 

Analysis  (LSA) ,  Life  Cycle  Cost  (LCC)  Analaysis,  business 
strategy,  contracting,  and  lessons  learned  from  past 
selection  committees.  In  addition,  dedicated  R&M  engineers 
would  be  invaluable.  Currently,  the  few  DPMLs  who  have 
served  on  source  selection  boards  stated  they  relied  on  the 
AFALC  staff  for  this  expertise,  but  due  to  timing  problems 
and  other  reasons,  the  expertise  could  not  always  be 
counted  on.  Contractors  are  aware  of  the  shortage  of 
experienced  evaluators  and  many  respondents  felt  the 
contractors  take  advantage  of  that. 

2.  Better  RFPs  to  the  Contractors.  Six 
respondents  recommended  that  the  RFPs  be  better  written  so 
that  the  contractors  can  better  understand  what  the  Air 
Force  is  really  requesting.  This  would  include  specifying 
the  logistics  test  results  the  contractor  must  demonstrate 


at  certain  milestones.  Too  often  the  RFPs  request 
unrealistic  or  questionable  goals  which  leads  contractors 
to  respond  with  proposals  stated  in  generalities  for 
"buying  into  contracts"  and  then  later  negotiating  away  the 
undefinable  goals.  The  RFPs  must  demand  facts  and  data 
that  the  contractor  can  be  made  to  uphold.  Some  of  the 
lessons  learned  from  writing  RFPs  could  be  transferred 
through  more  DPML  Conferences  (with  more  time  for  DPML 
interaction)  and  short  training  courses. 

3.  Increased  Program  Director  Guidance.  Five 
respondents  believed  that  the  best  way  to  weight  the 
logistics  issues  more  heavily  in  evaluating  the  RFPs  would 
be  through  increased  emphasis  on  the  Program  Directors 
being  accountable  for  logistics  issues.  Logistics  must  be 
considered  as  an  equal  partner  to  operational  performance 
when  the  final  recommendations  are  being  developed. 
Similarly,  the  up  channel  briefings  should  require  that 
logistics  issues  be  given  this  equal  weight  to  performance 
characteristics . 

Logistics  R&D.  All  nine  of  the  respondents  that 
offered  solutions  for  increasing  the  emphasis  on  developing 
and  applying  logistics  technology  recommended  a  few 
logisticians  be  placed  in  the  laboratories.  The  people 
selected  to  work  in  these  labs  would  have  to  have  an 
engineering  background  or  at  least  be  technically  minded, 
and  be  trained  in  knowing  where  to  find  funding  sources. 


The  tasks  outlined  by  the  DPMLs  for  these  logisticians 
fell  into  five  areas.  First,  they  would  need  to  need  to 
find  the  money  sources  for  the  laboratories  to  research 
various  logistics  R&D  issues.  Second,  they  would  have  to 
influence  or  change  the  mindset  of  the  laboratory  engineers 
to  look  at  these  supportability  issues  rather  than  the  more 
exciting  performance  issues.  They  might  include  a  tour  for 
the  engineers  to  visit  the  field  to  see  the  problems  and 
understand  the  issues.  Two  DPMLs  pointed  out  the  R&M 
engineeers  are  the  "black  sheep  of  their  community",  and 
until  this  perception  can  be  changed,  engineers  will  avoid 
supportability  engineering  issues.  Third,  they  would  need 
to  coordinate  and  develop  yearly  "think  tank"  sessions 
between  the  labs,  SPOs,  and  especially  the  users  or  MAJCOMs 
to  focus  on  new  areas  that  are  more  promising.  The 
follow-on  to  this  would  be  the  development  of  a  prioritized 
list  of  logistics  R&D  projects  to  compete  with  the  other 
R&D  projects  for  funding. 

Fourth,  technology  transfers  to  the  SPOs  would  need  to 
be  more  visible.  Almost  all  28  DPMLs  and  DOLs  pointed  out 
there  are  just  too  many  priority  issues  competing  for  the 
DPML's  time  for  him  to  explore  new  technologies.  Symposia 
are  interesting  but  most  of  the  attendees  would  not  need 
the  information  oased  on  the  phasing  of  their  programs. 
Therefore,  it  is  important  the  logisticians  in  the  labs 
seek  out  the  early  programs,  learn  about  them,  and 


then  offer  to  the  applicable  DPMLs  what  technology  might  be 
useful.  The  respondents  who  had  worked  with  the  Air  Force 
Coordinating  Office  for  Logistics  research  (AFCOLR)  had 
high  praises  of  their  help,  but  very  few  even  knew  of  the 
organization. 

Fifth,  they  would  need  to  help  direct  the  DPML  in  the 
best  ways  to  incorporate  new  logistics  technology.  This 
may  include  identifying  additional  funding  sources 
available  outside  the  SPO's  budget.  The  coordination  may 
identify  problems  in  the  project  for  the  Productivity, 
Reliability,  Availability,  and  Maintainability  Program 
Office  (PRAM)  to  solve  inorder  to  reduce  support  costs. 

This  may  also  include  helping  the  DPML  with  the  RFP 
phrasing  to  motivate  contractors  to  use  new  technologies  or 
to  provide  the  DPML  an  Air  Force  logistics  engineer  contact 
to  evaluate  the  contractors'  responses. 

Organizational  Structure.  Though  this  factor  was 
ranked  the  lowest  of  all  the  constraints,  it  raised 
probably  the  most  emotional  discussions.  Overall,  most  of 
the  respondents  felt  they  were  better  off  with  a  dual  chain 
of  command  than  directly  under  the  PD  and  ASD  as  long  as 
the  current  upper  level  emphasis  on  R&M  is  maintained. 

This  emphasis,  more  than  anything  else,  has  helped  make  the 
DPMLs'  jobs  easier  by  forcing  the  PDs  to  understand  the  R&M 
requirements  and  work  toward  their  resolution  since  they 
will  be  questioned  on  them  in  their  Program  Reviews.  The 
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DPMLs  felt  the  dual  chain  offered  them  additional 
flexibility  to  get  things  done  and  immunity  to  challenge 
the  PD.  However,  most  of  the  DPMLs  stated  they  have  never 
had  to  use  it  since  a  good  rapport  with  the  PD  was  usually 
enough  to  resolve  most  issues.  Most  felt  a  closer  tie  to 
the  PD  than  to  the  AFALC  chain  of  command.  However,  many 
of  the  respondents  brought  up  the  closely  related  OER 
issue.  Many  were  dissatisfied,  confused,  or  frustrated  by 
their  OER  chain  of  command.  During  the  course  of  these 
interviews  a  new  Air  Force  policy  was  announced  that  all 
OERs  will  receive  the  first  of  two  endorsements  from  the 
evaluator's  rating  official  in  order  to  prevent  inflated 
endorsement  levels.  However,  this  policy  creates  much 
confusion  for  the  DPMLs  since  for  them,  this  first  endorser 
will  be  in  a  completely  different  rating  chain  of  command. 
In  addition,  many  felt  they  worked  for  the  PD  but  he  never 
signed  their  OER  while  others  were  rated  by  neither  the 
AFALC  nor  the  product  division  headquarters.  Most  agreed 
that  some  kind  of  universal  policy  or  at  least  a  logical 
set  of  rules  must  be  established. 

Two  organizational  structure  recommendations  were 
initiated  almost  unanimously  by  the  respondents. 

1.  R&M  Engineers.  Twenty-three  of  the  28  respondents 
believed  the  R&M  engineers  within  the  SPOs  should  work 
directly  for  the  DPML  rather  than  under  the  engineering 
section . 
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2.  Staff  Assistance.  Twenty-four  of  the  28 
respondents  felt  the  AFALC  staff  should  be  reduced  or  their 
mission  redefined.  Most  stated  they  had  never  received  any 
really  beneficial  assistance  from  the  AFALC  staff  even 
though  AFALC  has  many  qualified  people.  They,  stated  that 
most  of  their  assistance  came  from  personnel  from  other 
SPOs,  from  the  ALCs,  and  from  AFLC  in  that  order.  The 
DPMLs  stated  they  were  frustrated  by  the  fact  they  worked 
long  hours  with  a  shortage  of  personnel,  many  of  them 
untrained,  and  desperately  needing  assistance  in  certain 
skills  while  the  staff  left  at  1630,  had  the  most  qualified 
technicians,  and  never  came  down  into  SPOs  to  provide  real 
assistance.  Most  of  the  DPMLs  and  DOLs  alike  stated  they 
were  unsure  what  functions  the  AFALC  staff  performed.  The 
degree  of  dissatisfaction  with  the  staff  was  clearly  the 
biggest  surprise  to  the  researcher  during  the  interviews. 
Its  breadth  and  intensity  exceeded  normal  organizational 
"griping".  Obviously,  this  perception,  whether  false  or 
true,  should  be  addressed  immediately. 

DPML  Authority.  DPML  Authority  was  rated  very  low  in 
impact  and  when  asked  what  solutions  the  respondents  felt 
would  help,  several  had  comments  but  only  eleven  felt  any 
solutions  were  even  necessary.  The  general  comments  tended 
to  fall  within  two  categories:  personal  skills  of  the 
DPML,  and  the  control  of  logistics  funds.  Most  of  the 
respondents  believed  that  the  DPML  had  enough  authority 


once  he  established  credibility  with  the  PD  through  his 
knowledge  of  logistics,  the  total  program,  and  his  ability 
to  keep  the  PD  advised  of  problems.  Most  felt  that  once  a 
DPML  knows  how  to  present  to  the  PD  an  R&M  initiative  in 
the  ways  that  it  will  benefit  the  PD,  the  DPML  will  have  no 
more  problems  with  authority.  Several  stated  the  PD  will 
often  test  the  new  DPMLs  before  trusting  them  to  speak  and 
act  for  the  SPO.  The  second  topic  commonly  commented  on 
was  whether  logistics  funds  should  be  "fenced".  This  is 
coding  and  managing  separately  certain  funds  so  that  the  PD 
cannot  use  those  funds  on  other  goals  or  needs.  Though  the 
DPMLs  were  split  on  this,  most  stated  it  had  little  bearing 
in  the  long  run  as  long  as  the  DPML  could  identify  his 
needs  and  get  those  items  on  contract  early.  A  few 
recommended  inflating  the  DPMLs  budget  estimates  of  known 
projects  in  order  to  provide  financial  flexibility  for 
later  unknown  projects  when  available  funding  may  be  harder 
to  attain.  Three  solutions  were  offered  for  improving  the 
DPML  powerbase:  first,  earlier  establishment  of  the  DPML 
office;  second,  improving  the  DPML  selection  process;  and 
third,  quality  controlling  the  existing  DPMLs. 

1.  Earlier  Establishment  of  DPML  Office.  Several 
stated  that  the  DPML  has  the  hardest  time  with  authority  in 
the  early  phases  of  the  program  when  he  could  be  having  the 
biggest  impact  on  R&M.  The  reason  for  this  problem  is 


often  that  the  DPML  and  any  technicians  he  may  be 
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authorized  are  not  being  assigned  as  early  as  in  the  other 
functions,  such  as  engineering.  Therefore,  it  was 
recommended  by  eight  respondents  to  assign  the  DPML  earlier 
and  assign  to  him  more  and  better  trained  technicians  in 
the  concept  exploration  and  demonstration  validation 
phases.  This  earlier  manning  would  provide  the  DPML  the 
ability  to  influence  early  decisions  and  to  have  the 
capability  to  conduct  the  analytical  assessments  that  give 
the  DPML  his  credibility  with  the  PD. 

2.  DPML  Selection  Process.  Two  recommended  the  DPML 
selection  process  should  be  more  selective  and  the 
applicants  should  be  interviewed  or  approved  by  product 
division  and  AFALC  senior  officers.  This  would  improve  the 
quality  of  incoming  DPMLs  and  would  improve  the  image  of 
the  position  to  both  the  DPMLs  and  to  others  in  the  SPO. 

3.  Quality  Control  of  DPMLs.  One  recommended  that 
AFALC  monitor  more  closely  the  performance  of  the  DPMi_,s  in 
the  SPOs.  Currently,  if  a  DPML  fails  to  take  the 
initiative  to  make  any  R&M  improvements,  the  AFALC  staff  is 
unaware  of  it  and  the  PD  is  often  insensitive  to  the 
logistics  matters  unless  they  have  a  current  impact. 

Trained  Personnel.  Ranked  as  the  constraint  with  the 
second  highest  impact,  22  of  the  28  respondents  offered 
suggestions  for  improving  the  definitely  perceived  lack  of 
training.  Of  all  the  issues,  the  DPMLs  and  DOLc  spent  more 
time  offering  specific  solutions  to  the  Trained  Personnel 


question  than  any  other  one.  As  many  of  them  stated,  even 
though  Requirements  Definition  and  RFP  Evaluation  Criteria 
could  really  affect  R&M  initiatives,  there  has  at  least 
been  big  improvements  in  those  areas  in  the  last  five  years 
and  the  Air  Force  is  working  on  even  more  improvements.  On 
the  other  hand,  the  overall  feeling  of  the  respondents  was 
that  the  lack  of  training  is  getting  worse  due  to  the 
growing  demand  for  specific  skills.  Therefore,  many  of 
them  felt  a  concerted  management  effort  in  this  area  would 
reap  big  benefits.  As  one  respondent  stated,  every  DPML 
must  become  a  dedicated  training  officer  for  his  people  or 
they  will  become  demoralized  and  request  a  transfer.  Their 
suggested  solutions  fell  within  four  natural  groupings: 
overall  management  of  the  career  field,  development  of  a 
comprehensive  training  program,  specific  course  curriculum 
needed,  and  alternatives  to  reduce  the  amount  of  training 
required . 

1.  Overall  Management  of  the  Career  Field.  Most  of 
the  respondents  felt  that  Acquisition  Logistics  should  be¬ 
come  a  separate  Air  Force  Specialty  Code  (AFSC)  or  at  least 
a  distinguishable  shredout  of  only  one  AFSC  by  a  prefix  or 
suffix.  Many  stated  this  would  help  toward  resolving  the 
training  problem  by  first,  identifying  what  individuals  are 
involved  with  acquisition  logistics;  second,  placing  them 
under  one  career  manager  who  would  be  responsible  for  the 
professional  development  of  the  career  field;  and  third, 
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allowing  one  management  evaluation  team  (MET)  to  determine 
manning  requirements.  Many  felt  that  identifying  the 
acquisition  logisticians  as  a  separate  specialty  would  help 
reduce  the  excessive  turnover  that  is  driving  much  of  the 
training  requirements.  It  is  perceived  that  a  lot  of  the 
turnover  is  caused  by  failure  to  recognize  acquisition 
logistics  as  a  unique  career  field  that  requires 
specialized  skills.  Establishing  one  career  manager  would 
allow  one  source  to  monitor  the  career  field  size,  develop 
a  career  master  plan,  and  establish  a  cycle,  or  natural 
succession  of  assignments  for  career  progression  that  would 
flow  logically.  This  career  flow  of  assignments  would 
provide  DPMLs  experience  from  the  ILSO  offices.  Air 
Logistics  Centers,  and  other  needed  background  areas 
depending  on  the  particular  SPO.  Currently,  the  DPMLs  are 
being  managed  by  several  different  career  monitors,  ie. 
supply,  maintenance,  program  manager,  transportation, 
logistics,  etc.,  that  move  them  in  and  out  of  the 
acquisition  arena  without  regard  to  what  the  other  career 
monitors  may  be  doing  to  resources  situated  in  the  same 
SPO.  Similarly,  by  having  them  identified,  these  skilled 
individuals  could  be  recalled  easier  after  other 
assignments.  Another  advantage  several  respondents 
mentioned,  would  be  the  increased  emphasis  to  coordinate 
people's  departure  assignments  to  significant  acquisition 
milestones  for  additional  program  stability.  Though  a 


difficult  policy  to  enforce,  several  felt  at  least  the 
requirement  for  upper  level  approval  of  transfers  would 
reduce  much  of  the  needless  turnover  and  training  of  new 
people.  A  few  stated  that  the  Air  Force  does  not  tolerate 
the  lack  of  a  training  program  in  other  AFSCs  such  as 
flying  and  maintenance,  yet  in  acquisition,  where  millions 
of  dollars  are  handled  in  daily  transactions,  there  is  no 
master  plan  for  selecting,  preparing  or  assigning  the 
personnel . 

2.  Development  of  a  Comprehensive  Training  Program. 
Most  of  the  respondents  felt  that  an  overall  training 
master  plan  should  be  developed  and  implemented  to  insure 
that  the  new  logisticians  receive  what  general  courses  are 
needed  and  that  there  is  a  logical  flow  into  follow-on 
refresher  courses.  The  consensus  was  that  there  should  be 
a  relatively  short  pipeline  into  the  SPO  consisting  of  a 
short  course  defining  their  mission,  the  acquisition 
organization  structures,  and  what  they  can  expect  and 
should  look  for  in  their  SPO.  Second,  the  individuals 
should  spend  approximately  two  weeks  within  their  SPOs  in 
order  to  understand  what  their  job  will  entail  and  then 
attend  an  acquisition  overview  course  such  as  the  AFIT 
Systems  100  course.  At  this  point  the  DPMLs  differed  on 
what  specific  type  training  courses  should  be  offered,  but 
they  all  agreed  it  should  be  specific  training  relative  to 
the  individual's  particular  SPO  rather  than  general 


education  or  on  areas  that  may  not  apply  to  that  SPO's 
situation. 

Some  of  the  respondents  were  frustrated  with  the 
management  of  training  slots.  The  respondents  from  the 
smaller  SPOs  stated  that  the  training  slots  are 
disproportionately  given  to  the  large  SPOs  because  they  can 
apply  more  political  pressure  to  get  the  slots.  No  one 
perceived  an  orderly  process.  Some  felt  that  the  overall 
training  office  should  be  within  AFALC. 

3.  Specific  Course  Offerings  Needed.  Though  there 
were  some  areas  of  general  agreement,  often  the  training 
desires  of  the  DPMLs  reflected  whether  they  were  from  large 
or  small  SPOs.  The  large  SPOs  tended  to  specialize  their 
people  more  and  thus  wanted  more  specific  logistics  skills 
taught.  The  smaller  SPOs  tended  to  want  more  general 
overview  courses.  Both  expressed  their  difficulty  in 
releasing  people  to  attend  programs,  but  the  smaller  SPOs 
were  affected  even  more  due  to  their  inherent  lack  of 
flexibility.  Both  felt  the  courses  offered  should  be  of 
shorter  duration  than  those  currently  offered.  Several 
expressed  a  desire  for  some  half  day  and  one  day  courses. 
They  felt  this  would  enable  more  people  to  attend.  Related 
to  the  difficulty  in  releasing  people,  several  expressed 
the  current  courses  have  too  much  wasted  time  in  general 
orientation,  welcoming  remarks,  etc.  They  felt  if  handouts 
and  readings  could  be  distributed  prior  to  arrival,  the 
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training  time  could  be  put  to  more  effective  use.  All  of 
the  respondents  felt  moving  the  Systems  100  course  to 
Randolph  AFB  was  a  mistake  due  to  the  additional  travel 
time.  Some  recommended  that  if  the  schools  cannot  increase 
the  number  of  their  courses,  maybe  the  experienced 
technicians  within  the  SPOs  could  teach  some  logistics 
skills  courses.  Two  advantages  would  be  that  first,  the 
material  would  tend  to  be  taught  at  a  more  realistic  and 
practical  level  and  second,  SPOs  would  offer  the  courses 
when  they  are  training  their  own  people.  Several  suggested 
there  should  be  a  close  coordination  between  the  SPOs,  the 
overall  AFALC  training  officer,  and  the  school.  The 
training  officer  should  be  able  to  require  the  specific 
course  offerings  and  dates  based  on  the  SPOs  needs.  Their 
needs  would  be  reflective  of  what  acquisition  phases  they 
are  entering. 

Many  of  the  DPMLs  expressed  what  courses  they  felt  were 
the  most  important.  Most  tended  to  feel  the  technicians 
needed  about  2/3  emphasis  on  logistics  skills  subjects  and 
about  1/3  emphasis  on  business  strategy  subjects.  They 
felt  the  split  ought  to  be  about  half  and  half  for  the 
DPMLs.  The  larger  SPOs  felt  they  could  OJT  the  logistic 
skills  but  needed  formal  courses  for  the  business 
acquisition  courses.  The  smaller  SPOs  felt  they  needed 
formal  courses  for  both.  In  logistics  skills,  the  subjects 
mentioned  most,  in  order,  were:  provisioning  for  supply 


and  support  equipment,  logistics  support  analysis  (LSA) , 
technical  order  management,  R&M,  Support  Equipment 
Requirement  Document  (SERD)  management,  life  cycle  costing 
(LCC) ,  tricks  of  the  trade,  and  quality  control  of 
contractors  and  vendors.  Several  commented  that  a 
checklist  approach  is  needed  in  the  courses  to  include 
points  of  contact  and  what  regulations,  directives,  and 
other  policies  should  be  referenced.  In  the  business 
acquisition  courses,  the  subjects  mentioned  most,  in  order, 
were  the  acquisition  process  (in  actuality  versus  theory) , 
contracting,  program  control  (to  include  the  various  kinds 
of  funds) ,  and  business  strategy. 

4.  Alternatives  to  reduce  the  Amount  of  Training 
Necessary.  Various  options  were  proposed  to  help  educate 
the  DPMLs  but  not  require  formal  training  courses.  The 
four  options  mentioned  most  were:  Pave  Tiger  or  assistance 
teams,  generic  checklists,  more  accessible  service 
contracts,  and  semi-annual  DPML  conferences. 

4.1  Assistance  Teams.  Ten  of  the  DPMLs  brought 
up  the  idea  of  developing  special  teams  to  go  into  the  SPOs 
when  projects  require  specialized  skills.  They  mentioned 
five  advantages  of  having  teams  over  the  current  method  of 
teaching  varied  groups  in  classrooms.  First,  since  these 
specialized  skills  are  often  used  only  once  in  a  program, 
it  is  inefficient  to  try  to  train  everyone  in  these  skills 
and  unrealistic  to  believe  they  will  learn  it  well  enough 


to  go  back  and  set  up  the  program  without  some  real 
problems.  Therefore,  the  teams  would  reduce  the  amount  of 
training  needed.  Second,  by  going  into  the  SPOs  to  set  up 
an  LSA  program  or  networking  chart  for  example,  all  the 
pertinent  personnel  in  that  SPO  could  be  instructed  on  how 
to  maintain  the  program  at  one  time  rather  than  just  the 
one  who  got  a  training  slot.  In  addition,  the  training 
would  occur  when  they  need  it  as  opposed  to  when  a  slot 
became  available.  Third,  the  team's  instructions  would  be 
tailored  for  that  SPO's  specific  needs.  Fourth,  the  team 
concept  would  have  a  much  better  chance  of  insuring 
successful  implementation  of  a  technique  since  the  team 
members  would  have  more  experience  to  apply  from  several 
projects.  Fifth,  this  would  be  a  more  efficient  use  of 
manpower  in  that  it  would  provide  the  flexibility  to  insert 
people  into  the  SPO  when  implementing  a  new  technique  and 
to  remove  them  when  the  workload  returns  to  normal.  This 
pool  of  experienced  technicians  would  be  especially 
important  to  SPOs  during  the  early  stages  when  their  ILSOs 
are  especially  undermanned  and  without  experienced  people. 

4.2  Generic  Checklists.  Seven  respondents 
advocated  using  generic  checklists  to  guide  the 
implementation  of  new  techniques.  These  general  checklists 
would  include  the  best  steps  for  setting  up  the  program, 
the  problem  areas  to  avoid,  reference  sources,  names,  phone 
numbers,  and  areas  of  specialities  of  people  they  could 
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contact  for  help,  and  lead  times  to  expect  in  accomplishing 
these  steps.  The  underlying  assumption  is  that  the 
checklists  provide  more  than  a  student  is  going  to  remember 
from  a  classroom  course  and  thus  eliminate  the  need  for 
some  formal  training. 

4.3  Services  Contracts.  Four  respondents 
suggested  that  many  of  the  projects  could  be  managed  more 
efficiently  through  services  contracts.  They  suggested 
that  the  firms  would  already  be  skilled  in  establishing  the 
projects,  would  provide  more  continuity  than  the  current 
SPO  manning  policies  provide,  and  could  be  held  accountable 
for  the  quality  of  their  work.  They  pointed  out  that  the 
biggest  drawback  to  this  solution  would  be  the  current 
guidance  policies  that  require  long  lead  times  before 
awarding  service  contracts  for  issues  that  often  have  short 
suspenses . 

4.4  DPML  Conferences.  Two  people  recommended 
that  semi-annual  DPML  conferences  would  be  a  good  way  to 
communicate  both  up  and  down  the  channel  the  important 
issues  in  acquisition  logistics.  In  addition,  both  felt  it 
would  also  be  beneficial  to  invite  the  logisticians  from 
the  contractors. 

Test  and  Evaluation.  Only  13  of  the  respondents 
offered  any  recommended  solutions  to  this  constraint.  Most 
of  the  respondents  admitted  they  had  little  or  no 
experience  working  with  test  and  evaluation  and  wished  they 
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knew  more  about  it.  Of  the  comments  received,  most  fell 
within  three  categories:  preparing  the  statement  of  work, 
managing  the  program,  and  working  with  the  Air  Force 
Operational  Test  and  Evaluation  Center  (AFOTEC) . 

1.  Preparation  of  Statement  of  Work.  The  majority  of 
the  respondents  indicated  that  the  most  important  issue  was 
clearly  identifying  in  the  statement  of  work  the  R&M 
specifications  the  contractor  should  demonstrate.  The  DPML 
must  also  establish  test  data  points.  To  further  insure 
the  contractor  fulfills  these  testing  specifications,  the 
contract  should  contain  significant  penalties  for  failures 
and  incentive  bonuses  (particularly  the  ones  that  designate 
some  of  the  bonus  to  the  contractor's  logistics  personnel) 
for  surpassing  the  specifications.  Seven  of  the  DPMLs 
stated  people  experienced  in  test  and  evaluation  should  be 
coordinated  with  during  the  development  of  the  RFP  and 
SOW.  They  suggested  both  the  assistance  teams  mentioned 
earlier  and  AFOTEC  as  the  best  sources  of  expertise. 

2.  Management  of  the  Program.  Once  the  SOW  is 
completed,  five  DPMLs  remarked  on  the  importance  of 
attaining  dedicated  resources  from  the  PD.  They  stated  the 
trend  that  program  review  briefings  have  started 
questioning  PDs  on  what  specific  testing  data  has  been 
accumulated  has  helped  the  DPMLs  more  than  anything  else. 

A  few  also  recommended  that  DPMLs  be  educated  to  coordinate 
as  early  as  possible  for  dedicated  testing  times, 


conditions,  and  places  for  the  logistics  tests.  The  tests 
should  resemble  the  expected  operational  environment  as 
much  as  possible.  In  addition,  two  stated  that  AFOTEC  has 
a  tendency  to  place  all  logistics  tests  in  the  undesirable 
hours.  This  tends  to  discourage  the  opportunity  of  having 
key  personnel  attend  these  events.  DPMLs  must  also  be 
educated  to  limit  the  contractors  as  to  what  conditions, 
additional  supplies,  and  personnel  they  use  to  influence 
the  tests.  Testing  and  Evaluation  checklists  and  lessons 
learned  are  essential  to  alert  the  DPMLs  of  these  and  other 
possible  pitfalls. 

3.  AFOTEC  Working  Relations.  Four  respondents  stated 
that  AFOTEC  needs  to  maintain  its  independent  role  but  not 
to  the  degree  that  it  cannot  help  the  SPOs .  These  four 
felt  AFOTEC  often  perceives  of  itself  as  the  IG  and  only 
critiques  the  SPOs  after  test  dollars  have  been  wasted 
rather  than  helping  the  SPOs  prepare  and  avoid  mistakes. 
However  in  contrast,  two  stated  that  AFOTEC  had  served  on 
their  SPO  team  to  establish  testing  procedures  and  had 
helped  solve  several  problems. 

Suggested  Solutions  to  Respondent  Generated  Constraints 

Research  question  three  identified  15  constraints  that 
the  respondents  initiated.  Though  some  of  their  constraints 
and  solutions  overlap  with  the  original  seven  already 
covered.  Table  VIII  summarizes  the  suggested  solution  each 
respondent  gave  following  their  perceived  constraint. 


TABLE  VIII 


Suggested  Solutions  to  Respondent  Generated  Constraints 


Constraint 


1.  PD  interest  in 
R&M  issues 


2 .  Manpower 

Authorizations 


3.  Manning 

Continuity 


4.  Assignment 
of  R&M 
Engineers 


5 .  Manning  of 
Basket  SPOs 


6 .  ALC  Support 


Solution 


Continued  emphasis  by  upper  level 
management  to  hold  the  PDs 
accountable  for  the  R&M  issues  during 
their  program  review  briefings. 

One  Management  Engineering  Team  (MET) 
needs  to  be  in  charge  of  determining 
manning  authorizations  and  one 
personnel  office  in  charge  of  filling 
the  justified  slots. 

Freeze  assignments  of  both  civilian 
and  military  personnel  to  milestones 
or  project  completions.  This  would 
provide  continuity  and  help  insure 
that  important  management  initiatives 
are  fully  carried  out  and  managers 
are  held  more  accountable  for  their 
actions.  Senior  officers  could 
review  and  waiver  any  early  moves. 

R&M  engineers  should  work  directly 
under  and  be  evaluated  by  the  DPML . 
Their  duty  location  should  depend  on 
the  SPO,  but  either  be  collocated 
with  the  design  engineers  or  in  the 
DPML  office. 

Basket  SPOs  should  be  evaluated 
before  assigning  inexperienced  people 
to  them.  The  majority  of  the 
inexperienced  should  be  assigned  to 
the  larger  SPOs  which  are  more 
capable  of  training  them. 

AFLC  and  the  ALCs  must  take  a  more 
active  role  in  the  early  acquisition 
phases  to  ensure  contractors  are 
providing  supportable  and  supported 
systems.  The  excuse  of  lacking  TDY 
funding  is  not  justified  in 
comparison  to  the  value  of  early 
expertise . 
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TABLE  VIII  (continued) 


Constraint 


7.  Logistics 
Experience 


8.  Lack  of  Real 
Staff 

Assistance 


9 .  Funding 
Sources 


10.  Contractor 
Organization 
Structure 


11.  DPML 

Assertiveness 


Solution 


Acquisition  logisticians  should  have 
first  completed  some  form  of 
logistics  assignment  (preferably  with 
a  system  similar  to  the  SPO's 
product)  before  being  assigned  to  the 
SPO.  Once  in  the  SPO,  the  product 
divisions  should  use  TDYs ,  briefings, 
or  other  forms  to  keep  the 
logisticians  familiar  with  the 
product  and  its  inte  ded  environment. 

AFALC  senior  officers  need  to 
encourage  their  staff  specialists  to 
work  with  SPOs  requesting  assistance 
for  projects  requiring  specific 
skills.  They  should  stay  in  the  SPO 
for  however  long  it  takes  to  get  the 
project  going,  thus  becoming 
committed  to  its  success. 

DPMLs  should  be  either,  taught  through 
a  symposium  or  checklist  type 
document  what  types  of  funds  are 
available,  how  to  acquire  them,  and 
which  documents  are  available  to 
track  funds. 

DPMLs  and  PDs  must  insure  the 
contractor's  logistics  and 
engineering  offices  maintain  equal 
status  and  that  there  is  good 
communication  between  them.  Starting 
at  the  guidance  conference,  the  DPML 
and  PD  must  demonstrate  to  the 
contractor  they  expect  equal  emphasis 
to  be  placed  on  logistics  issues. 

This  will  aid  the  contractor's 
logisticians  and  will  heighten  the 
company's  perceptions  of  their  ideas 
and  status . 

Program  Directors  and  DOLs  need 
to  insure  that  their  DPMLs  do  not 
accept  everything  a  contractor  states 
and  that  the  issues  are  actively 
challenged  by  the  DPMLs. 


TABLE  VIII  (continued) 


Constraint 


12.  Conflicting  & 
Confusing 
Guidance 


13.  Sole  Source 
Requirements 


14.  Different 
Leadership 
Perspectives 


15.  Management 
of  Joint 
Programs 


Solution 

A  "task  force"  should  be  established 
to  tailor  re-define,  and  update  DOD, 
Air  Force,  AFSC,  AFLC,  AFALC,  and 
product  division  regulations, 
directives,  and  policies  concerning 
acquisition  and  logistics.  In 
addition,  provisions  for  innovative 
techniques  should  be  allowed  rather 
than  foregoing  cost  savings  due  to 
paperwork  requirements. 

DOD  should  ease  sole  sourcing 
requirements  or  streamline  the 
procedures  in  order  that  cost  saving 
R&M  initiatives  can  be  implemented 
into  contracts  expeditiously. 

Policy  letters  and  other  forms  of 
communication  to  the  SPO  personnel 
should  be  more  in  the  form  of  a 
coordinated  agreement  between  either 
AFLC  and  AFSC,  or  AFALC  and  the 
product  divisions. 

Though  it  may  be  tough  politically, 
the  Air  Force  should  baseline  all 
joint  programs  and  then  place  each 
program  under  one  service  using  only 
that  service's  procedures.  Having 
only  one  service  manage  a  program 
will  result  in  both  cost  and  time 
savings . 


V.  Conclusions  and  Recommendations 


During  the  development  of  this  research  project,  it  was 
assumed  that  the  DPMLs  and  DOLs  of  the  major  programs  would 
have  had  more  DPML  experience  and  experience  during  the 
early  acquisition  phases  than  they  did.  Because  a  few  had 
never  worked  with  any  program  during  its  concept 
exploration,  demonstration  validation,  or  early  portions  of 
full  scale  development,  these  individuals  had  difficulty  in 
providing  specific  recommended  solutions  to  factors  such  as 
Requirements  Definition  and  RFP  Evaluation  Criteria. 

However,  the  researcher  feels  this  had  little  effect  on  the 
rating  results  or  other  recommendations  for  three  reasons. 
First,  the  statistical  tests  performed  showed  little 
difference  in  the  ratings  of  the  experienced  and  less 
experienced.  Second,  most  programs  had  several  on-going 
modifications  and  changes  that  allowed  the  DPMLs  to 
experience  to  a  degree  many  of  the  same  steps  and 
frustrations  of  programs  in  the  early  phases.  Third,  being 
on  the  end  of  a  program  often  allows  one  the  advantage  of 
hindsight  and  to  understand  what  should  have  been  done 
earlier  in  the  program’s  life.  Overall,  the  researcher 
believes  the  ratings  and  recommended  solutions  presented  are 
valid  and  present  a  formally  organized  assessment  of  what 


Research  Question  One 

The  first  objective  was  to  identify  "How  do  the  DPMLs 
rank  order  seven  major  constraints  as  to  their  impact  on 
impairing  the  DPMLs'  ability  to  influence  a  system's  design 
for  improved  supportability?" 

The  factors  were  rated  by  the  respondents  on  graphic 
rating  scales  and  then  ranked  by  their  mean  measurements  on 
the  scales.  The  overall  ranking  of  each  factor  was  very 
consistent  within  all  three  experience  level  groupings  of 
the  respondents.  In  addition,  the  mean  measurements  of  all 
the  factors  were  clearly  separated  thus  making  distinctions 
between  rankings  very  definite. 

The  rankings,  as  depicted  in  Table  IV,  closely  resembled 
the  results  of  the  Hull  and  Lockhart  Study  in  1982.  Though 
some  of  the  factors  rated  in  that  study  were  different  and 
the  similar  ones  were  phrased  differently,  there  were  close 
similarities  showing  a  consistency  over  time  of  the  DPMLs' 
perceptions.  The  Hull  and  Lockhart  study  rated  Logistics 
Design  Goal  Definition,  which  was  defined  similarly  to 
Requirements  Definition,  also  as  the  number  one  factor. 

Their  study  rated  Goal  Conflict,  defined  similarly  to  RFP 
Evaluation  Criteria,  as  the  second  factor?  RFP  Evaluation 
Criteria  was  ranked  third  in  this  study.  Conversely,  their 
study  rated  Logistics  Skills,  defined  similarly  to  Trained 
Personnel,  as  third  while  it  was  rated  second  in  this 
study.  This  study  tended  to  rate  the  lack  of  DPML 


Authority  relatively  lower  than  in  the  previous  study  which 
probably  reflects  the  increased  emphasis  R&M  issues  have 
received  from  senior  officers  in  the  last  three  years. 
Though  the  ratings  of  the  two  studies  cannot  be  compared 
directly  due  to  the  differences  in  the  two  studies,  the 
overall  similarities  tend  to  strengthen  the  statistical 
analyses  in  this  study. 

The  researcher  oelieves  the  rankings  accurately 
represent  the  impact  of  the  factors  as  perceived  by  the 
DPMLs  and  DOLs  of  major  weapon  systems. 

Research  Question  Two 

The  second  objective  was  to  identify;  "Do  the  DPMLs 
significantly  differ  in  their  perception  of  the  rank 
ordering  of  the  seven  constraints  by  product  division?" 

The  ratings  of  the  DPMLs  and  DOLs  of  the  major  systems 
within  ASD  and  ESD  were  compared.  Statistical  tests  at  the 
.05  significance  level  were  performed  comparing  all  three 
experience  level  groupings  within  the  two  product 
divisions.  There  were  no  statistically  significant 
differences  in  any  of  their  ratings.  As  shown  in  Table  VI, 
the  rankings  of  all  the  respondents  differed  only  slightly 
between  the  two  product  divisions  by  the  reversal  of  the 
DPML  Authority  and  Test  and  Evaluation  factors  for  the 
fifth  and  sixth  positions. 

The  researcher  believes  that  because  of  the  lack  of  any 
statistical  differences  between  the  two  product  divisions, 
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regardless  of  experience  levels  analyzed,  it  can  be 
reasonably  assumed  that  there  are  no  statistically 
significant  differences  in  the  perceptions  of  the  DPMLs  of 
any  of  the  product  divisions. 

Because  of  the  differences  in  experience  levels, 
statistical  tests  were  also  conducted  to  identify  if  any  of 
the  other  demographic  categories  differed  significantly  in 
their  ratings.  The  results  showed  rank,  time  as  a  DPML,  or 
early/ late  acquisition  experience  had  little  bearing  on  the 
rankings.  The  comparison  of  those  with  less  than  three 
years  in  acquisition  versus  those  with  more,  however, 
showed  the  less  experienced  rated  the  factor  of  DPML 
Authority  significantly  higher  and  Test  and  Evaluation  much 
lower  than  the  more  experienced  respondents.  There  was 
also  an  overall  tendency  that  the  more  experienced  and 
those  with  early  acquisition  phase  backgrounds  to  place 
more  emphasis  on  trained  personnel. 

Research  Question  Three 

The  third  objective  was  to  identify,  "What  other 
constraints  do  the  DPMLs  perceive  as  significant?" 

Fifteen  additional  constraints  were  identified  and 
grouped  into  six  areas:  Program  Director  priorities, 
manning,  lack  of  assistance,  funds  control,  contractor 
related,  policy  and  guidance,  and  management  procedures  of 
joint  programs.  These  constraints  are  described  in  Chapter 
IV.  The  majority  are  derived  from  the  overwhelming 


consensus  that  the  Acquisition  Logistics  career  field  needs 
to  be  managed  separately.  They  provide  a  good  "snapshot" 
view  of  what  additional  constraints  were  on  the  minds  of 
the  DPMLs  on  the  days  they  were  interviewed. 

Research  Question  Four 

The  fourth  objective  was  to  identify,  "What  various 
solutions  do  the  DPMLs  propose  to  reduce  the  impacts  of  the 
constraints  in  questions  one  and  three?" 

The  recommendations  were  summarized  and  paraphrased  as 
closely  as  possible.  Soliciting  the  recommendations  took 
the  largest  percentage  of  time  during  the  interviews  and 
also  in  presenting  the  results.  However,  they  provide  the 
greatest  opportunity  for  applied  value.  Though  only  a  few 
may  have  mentioned  some  solutions,  it  is  not  inferred  that 
the  others  did  not  agree,  but  that  they  merely  may  not  have 
thought  of  it  at  the  time. 

As  could  be  expected,  the  interviewer  improved  during 
the  course  of  the  interviews  in  his  ability  to  administer 
the  interviews  and  in  his  understanding  of  the  subject 
matter.  The  likely  result  of  this  learning  process  was 
that  better  recommendations  were  gathered  from  the  later 
interviews.  It  was  not  in  the  scope  of  this  research  to 
evaluate  the  recommended  solutions.  However,  it  appears 
some  recommendations  such  as  the  development  of  user 
conferences,  the  development  of  management  tools  to 
operationalize  reliability  figures,  the  development  of  a 


source  selection  cadre,  the  assignment  of  DPMLs  to 
laboratories,  changing  the  control  of  R&M  engineers, 
establishing  a  separate  management  of  the  career  field  and 
its  training  requirements,  development  of  assistance  teams 
for  specialized  skills,  and  the  use  of  generic  checklists 
show  definite  promise. 

Recommendations 

The  researcher  offers  two  recommendations  for  further 
study  based  on  the  findings  of  this  research.  First, 
reaccomplish  this  study  in  a  few  of  the  acquisition 
organizations  within  the  Army  and  Navy.  Though  this  study 
could  be  generalized  to  be  repres'entative  of  all  Air  Force 
product  divisions,  it  would  be  important  to  discover  if  the 
same  issues  are  significant  in  the  other  services.  If  not, 
what  procedures  are  being  used.  If  the  same  problems  are 
evident  throughout  the  DOD,  the  organizational  level 
responsible  for  initiating  a  resolution  will  be  different. 

A  second  recommendation  would  be  for  further  research 
to  be  conducted  on  the  recommended  solutions  offered  by  the 
respondents.  Each  recommendation  should  be  researched  to 
see  how  much,  if  any,  additional  funding,  manning,  and 
other  resources  would  be  required,  their  feasibility,  and 
compare  these  costs  to  what  additional  benefits  could  be 
logically  expected.  Those  recommendations  which  prove  to 
be  feasible  and  cost  effective  should  be  briefed  to  the 
appropriate  staff  for  recommended  implementation. 
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Appendix  A:  Interview  Form 
SURVEY  INFORMATION  SHEET 


Hypothetical  Case 

The  Air  Staff  is  interested  in  finding  out  what  the 
DPMLs  believe  are  their  most  significant  constraints  in 
implementing  Reliability  and  Maintainability  into  new 
weapon  system  designs.  Therefore,  they  developed  a  list  of 
possible  factors  which  may  be  contributing  to  the  problem. 
They  have  asked  you  to  apply  your  personal  experience  and 
professional  judgement  to  help  them  determine  the  order  of 
significance  of  these  7  factors. 

Instructions 


1.  Please  read  the  entire  list  of  factors  before  rating 
the  impact  of  any  of  them. 

2.  Select  the  factor  you  feel  has  the  MOST  IMPACT,  mark  it 
on  the  scale,  and  place  the  letter  of  the  factor  next 
to  the  mark.  (Note  the  scale  will  allow  you  to  rate 
not  only  the  order  of  the  factors,  but  also  the 
difference  in  significance  by  their  relative  position 
on  the  scale.) 

3.  Continue  rating  the  factors  until  you  have  marked  all 
of  the  factors  on  the  scale. 

4.  If  you  feel  two  or  more  factors  are  equally  important, 
use  a  single  mark,  but  please  be  sure  all  letters  are 
accounted  for. 


Example  of  Rating  Process;  What  is  the  relative  impact  of 
these  factors  on  your  personal  budget? 


A. 

Car-Loan 

B. 

Entertainment  Expense 

C. 

Home  Mortgage 

HIGH 

IMPACT 

— 

-  c 

D. 

Groceries 

SOME 

IMPACT 

— 

-  A 

- 

-2> 

LOW 

IMPACT 

Z3 

Schedule  No. 


SURVEY  ANSWER  SHEETS 

Personal  Data 

What  is  your  present  grade? 


A. 

0-1 

to 

0-3 

D. 

GS-9 

to  GS-12 

B. 

0-4 

to 

0-5 

E. 

GS-13 

to  GS-14 

C. 

0-6 

or 

higher 

F. 

GS-15 

or  higher 

What  is  the  total  time  you  have  worked  in  system 
acquisition? 

A.  Less  than  one  year  C.  Three  to  five  years 

B.  One  or  two  years  D.  More  than  five  years 

How  long  have  you  worked  as  the  DPML  in  a  SPO? 

A.  Less  than  6  months  C.  One  to  2  years 

B.  Six  months  to  1  year  D.  More  than  2  years 

How  long  have  you  worked  as  a  DPML  in  programs  during 
their  early  concept  exploration  or  demonstration 
validation  phases? 

A.  Less  than  6  months  C.  One  to  2  years 

B.  Six  months  to  1  year  D.  More  than  2  years 

In  which  product  division  are  you  assigned? 


CONSTRAINTS  TO  IMPLEMENTING  R&M  INITIATIVES 


A.  REQUIREMENTS  DEFINITION;  Inadequate  definition  of 
logistics  design  parameters  in  early  program 
documentation.  (Includes  such  issues  as  ambiguous 
statements  of  need  and  the  difficulty  of  defining 
specific  logistics  goals  in  early  program  plans  and 
constrast  statements  of  work.) 

B.  RFP  EVALUATION  CRITERIA;  Inadequate  weighing  of 
logistics  issues  in  evaluating  RFPs.  (Includes  such 
issues  as  the  composition  of  the  source  selection 
committees,  failure  to  convince  contractors  that  R&M 
issues  are  equal  to  cost  and  performance  issues,  lack 
of  award  fee  contracts  on  R&M  issues,  and  the 
reluctance  to  give  more  weight  to  proposals  with 
innovative  approaches  to  supportability . ) 

C.  LOGISTICS  R&D ;  Insufficient  emphasis  on  developing  and 
applying  logistics  technology.  (Includes  such  issues 
as  the  lack  of  communication  from  the  Coordinating 
Office  for  Logistics  Research  and  Development  and  the 
Logisitics  and  Human  Factors  Division  of  the  Human 
Resources  Laboratory.) 

D.  ORGANIZATIONAL  STRUCTURE;  Dual  chain  of  command  splits 
DPML  loyalty.  (Includes  such  issues  as  inhibiting 
DPMLs  from  being  part  of  the  logistic  and  acquisition 
teams,  and  inhibiting  communications  with  the  SPO  and 
logistic  agencies.) 

E.  DPML  AUTHORITY:  The  lack  of  delegated  or  assumed 
decision  making  authority.  (Includes  such  issues  as 
control  of  logistics  fund  expenditures  and  the  ability 
to  influence  design  trade-off  decisions  when  there  is  a 
conflict  with  performance  oriented  goals.) 

F.  TRAINED  PERSONNEL:  Failure  to  employ  experienced  and 
trained  logisticians.  (Includes  such  issues  as  being 
assigned  mostly  inexperienced  personnel  officers,  rapid 
turnover,  and  the  lack  of  meaningful  training  in  LSA, 
acquisition  strategies,  incentives,  R&M  issues,  and 
systems  analysis.) 

G.  TEST  AND  EVALUATION:  Inadequate  T&E  for 
supportability.  (Includes  such  issues  as  planning, 
funding,  testing  based  on  operational  requirements, 
early  testing  under  realistic  conditions,  and  clear 
definition  of  government  and  contractor 
responsibilities . ) 
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All  factors  will  be  ranked  on  this  one  scale.  Please 


indicate  your  feelings  about  the  impact  each  listed  factor 
has  on  R&M  in  a  hypothetical  program  office.  Make  sure 
each  hash  mark  is  identified  by  the  letter  of  the  factor  it 
represents . 


SIGNIFICANT  IMPACT 


SOME  IMPACT 


LOW  IMPACT 
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Schedule  No. 


What  other  factors  do  you  consider  significant? 


If  you  were  "Chief  of  Staff  for  a  day,"  what  solutions 
would  you  propose  to  lessen  the  impact  of  the  seven 
measured  constraints  and  any  of  the  above  factors. 
Please  be  as  specific  as  possible  by  identifying  action 
agencies,  and  other  pertinent  details. 


Appendix  B:  Data  Inputs  From  Respondents 


Below  are  the  data  inputs  for  groups  one,  two  and 
three.  Group  one  contains  all  the  responses  while  groups 
two  and  three  are  just  the  more  experienced  respondents 
from  group  one.  Each  line  signifies  the  responses  of  one 
interview.  The  first  five  digits  are  the  answers  to  the 
demographic  questions .  The  next  seven  groupings  are  the 
measurements,  in  millimeters,  of  each  respondent's  ratings 
of  the  factors  from  the  graphic  rating  scale. 
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Appendix  C:  SPSS  Program  Code 


RUN  NAME 
PRINT  BACK 
VARIABLE  LIST 
INPUT  MEDIUM 
N  OF  CASES 
INPUT  FORMAT 
VAR  LABELS 


MISSING  VALUES 

FREQUENCIES 

OPTIONS 

STATISTICS 

READ  INPUT  DATA 

T-TEST 

ONEWAY 

ONEWAY 

ONEWAY 

ONEWAY 

♦RECODE 

T-TEST 

♦RECODE 

T-TEST 

♦RECODE 

T-TEST 

♦RECODE 

T-TEST 

13.19.54. UCLP,CA, 


THESIS 
CONTROL 
Q1  TO  Q 12 
CARDS 
UNKNOWN 

FIXED (5F1,1X,7F3.1) 

Q1 ,GRADE/Q2 ,TIME  IN  ACQ/Q3 ,TIME  AS 
DPML/Q4 ,  TIME  IN  CE  OR  DV/Q7,RFP 
EVAL  CRITERIA/Q8, LOGISTICS 
R&D/Q9 , ORGAN.  STRUCTURE/ Q10 ,  DPML 
AUTHORITY/ AL/ 

Q1  TO  Q12 (0) 

GENERAL=Q1  TO  Q5 

3,8 

ALL 

GROUPS=  Q5/VARIABLES=Q6  TO  Q12/ 

Q6  TO  Q12  BY  Ql(l,6) 

Q6  TO  Q12  BY  Q2(l,4) 

Q6  TO  Q12  BY  Q3(l,4) 

Q6  TO  Q12  BY  Q4(l,4) 

Q1  (2  =  1)  (3  =  2)  (4  =  1)  (5  =  1)  (6  =  2) 

GROUPS =Ql/VARIABLES=Q6TO  Q12/ 

Q2(2  =  l)  (3  =  2)  (4  =  2) 

GROUPS =Q2/ VAR I ABLES=Q6  TO  Q12/ 

Q3  ( 2  =  1 )  (3  =  2)  (4  =  2) 

GROUPS =Q3/VARIABLES=Q6  TO  Q12/ 

Q4 (2  =  1)  (3  =  2)  (4  =  2) 

GROUPS =Q4/ VARIABLES =Q6  TO  Q12/ 


N1706H3, 


0 . 176KLNS . 


Appendix  D:  Sample  Statistical  Test  For  Evaluating  Each 
Factor  Between  ASD  &  ESD 


VARIABLE 

09 

ORGAN.  STRUCTURE 

NUMBER 

OF  CASES 

MEAN 

STAN0AR0 

DEVIATION 

STANDARD 

ERROR 

CROUP 

1 

16 

23.9375 

21.019 

5.255 

GROUP 

2 

12 

18.5833 

13.747 

3.969 

POOLED  VARIANCE  ESTIMATE  SEPARATE  VARIANCE  ESTIMATE 


F 

2-TAIL 

T 

DEGREES  OF 

2-TAIL 

T 

DEGREES  OF 

2-TAIL 

VALUE 

PR  OB. 

VALUE 

FREEOCM 

PROB. 

VALUE 

FREEDOM 

PKOB. 

2.34 

.  16C 

.77 

26 

.450 

.81 

25.62 

.424 

The  above  portion  of  the  computer  printout  demonstrates 
the  statistical  analyses  performed  on  each  of  the  seven 
rated  factors  when  compared  between  ASD  and  ESD.  Following 
is  a  sample  of  the  statistical  tests  the  researcher 
performed  on  each  of  the  seven  factors  using  the  computer 
information.  This  particular  sample  showed  there  was  no 
significant  statistical  difference  in  the  ratings  given  by 
the  ASD  respondents,  group  one,  when  compared  to  the  ESD 
respondents,  group  two,  concerning  the  rated  factor. 
Organization  Structure. 


Reject  if  F  >  F  a/ 2 
2.34  >  3.33 


Reject  if  t  >  t  a/2 
.77  >  2.07 


Rej ic  H0:  Use  Pooled 
Variance 


Do  not  reject  H0: 

No  Significant  Difference 


Appendix  E:  Sample  Statistical  Test  For  Evaluating 
Demoqrahic  Category 


VARIABLE  Q8 

BY  Q4 

logistics  r&o 

TINE  IN  CE  OR 

ov 

ANALYSIS 

OF  VARIANCE 

SOURCE 

DaF  a 

SUM  CF  SQa 

MEAN  SO. 

F  RATIO  F  PROB 

BETWEEN  GROUPS 

3 

1307.875 

435.958 

1.012  . 4C45 

WITHIN  groups 

24 

1 Q  335. 090 

430.629 

TOTAL 

27 

1 1642.984 

THESIS 

85/08/01.  13.43.35.  PAGE  4b 

FILE  -  NONANE 

(CREATEO  -  85/08/01) 

VARIABLE  Q9 

BY  04 

ORGAN.  STRUCTURE 

TINE  IN  CE  OR  QV 

ANALYSIS 

OF  VARIANCE 

SOURCE 

0.F  a 

SUM  CF  SO. 

MEAN  SO. 

F  RATIO  F  PROB 

BETWEEN  GROUPS 

3 

2081.352 

693.784 

2.441  .0889 

WITHIN  GROUPS 

24 

6821.077 

284.21 2 

TOTAL 

2/ 

8902.429 

The  above  portion  of  the 

computer 

printout 

demonstrates 

the  statistical 

analyses  performed  on 

each  of  the  seven 

factors  by  each 

of  the  four 

demographic  categories  of 

grade,  time  in 

acquisition , 

time  as  a 

DPML,  and 

time  in  the 

early  acquisition  phases.  This  particular  sample  tests  the 
variance  is  in  responses  to  how  logistics  R&D  and 
Organizational  Structure  were  rated  between  each  early 
acquisition  experience  level  and  within  each  of  those 
experience  levels.  Below  is  the  statistical  test  the 


researcher  performed  on  each  factor  using  the  computer 
information. 


Reject  if  F  >  f'.05<vlv2) 

1.012  >  2.8 

Do  not  reject:  No  Significant  Difference. 


rWjvf  vO1-’ v 


Appendix  F:  Sample  Statistical  Test  For  Evaluatinc 
Each  Factor  By  Experience  Levels 


VARIA8LE 

08 

LOGISTICS  R&D 

NUMHER 

OF  CASES 

MEAN 

STANDARD 

DEVIATION 

STaNDAKO 

ERROR 

GROUP 

1 

23 

41.3043 

21.074 

4.394 

GROUP 

2 

5 

53.4000 

1  7.827 

7.972 

POOLED  VARIANCE  ESTIMATE  SEPARATE  VARIANCE  ESTIMATE 

F  2-TAIL  T  OEGREES  OF  2-TAIL  T  DEGREES  OF  2-TAIL 

VALUE  PROB.  VALUE  FREEOON  PROB.  VALUE  FREEDOM  PROB. 

1.40  .819  -1.10  26  .24S  -1.53  6.69  .226  j 

The  above  portion  of  the  computer  printout  demonstrates 
the  statistical  analyses  performed  on  each  of  the  seven 
rated  factors  when  comparing  the  less  experienced  in  each 
demographic  category,  or  group  one,  to  the  more  experienced 
respondents,  or  group  two.  This  particular  sample  is 
comparing  the  responses  to  the  Logistics  R&D  factor  by 
those  with  less  than  three  years  in  acquisition  with  those 
with  three  or  more  years.  Below  are  the  statistical  tests 
the  researcher  performed  on  each  factor  using  the  computer 
information. 

Reject  if  F  >  F  a/2  Reject  if  t  >  t  a/2 

1.4  >  8.53 


Use  Pooled  Variance 


-1.9  <  -2.056 
No  Significant  Difference 
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